PBAC: A pathway‐based attention convolution neural network for predicting clinical drug treatment responses

Precise and personalized drug application is crucial in the clinical treatment of complex diseases. Although neural networks offer a new approach to improving drug strategies, their internal structure is difficult to interpret. Here, we propose PBAC (Pathway‐Based Attention Convolution neural network), which integrates a deep learning framework and attention mechanism to address the complex biological pathway information, thereby provide a biology function‐based robust drug responsiveness prediction model. PBAC has four layers: gene‐pathway layer, attention layer, convolution layer and fully connected layer. PBAC improves the performance of predicting drug responsiveness by focusing on important pathways, helping us understand the mechanism of drug action in diseases. We validated the PBAC model using data from four chemotherapy drugs (Bortezomib, Cisplatin, Docetaxel and Paclitaxel) and 11 immunotherapy datasets. In the majority of datasets, PBAC exhibits superior performance compared to traditional machine learning methods and other research approaches (area under curve = 0.81, the area under the precision‐recall curve = 0.73). Using PBAC attention layer output, we identified some pathways as potential core cancer regulators, providing good interpretability for drug treatment prediction. In summary, we presented PBAC, a powerful tool to predict drug responsiveness based on the biology pathway information and explore the potential cancer‐driving pathways.

convolution layer and fully connected layer.PBAC improves the performance of predicting drug responsiveness by focusing on important pathways, helping us understand the mechanism of drug action in diseases.We validated the PBAC model using data from four chemotherapy drugs (Bortezomib, Cisplatin, Docetaxel and Paclitaxel) and 11 immunotherapy datasets.In the majority of datasets, PBAC exhibits superior performance compared to traditional machine learning methods and other research approaches (area under curve = 0.81, the area under the precision-recall curve = 0.73).
Chemotherapy and immunotherapy are fundamental methods in cancer treatment.Chemotherapy, with its profound history, is a foundational therapy for cancer, while immunotherapy represents a novel approach to cancer treatment. 1,2Both of these treatments are essential in clinical oncology.Nevertheless, they share a common limitation: they exhibit significant heterogeneity in treatment responses, with numerous patients failing to respond and encountering severe side effects from the therapies. 3Therefore, identifying a patient's potential response to drug treatment is extremely important for precision medicine in clinical practice.
A major challenge in the research of drug therapeutic response prediction is its practical application in the clinical context. 4Despite the significant advancements made by models such as developed by Ding et al., 5 AutoBorutaRF 6 and DeepDR 7 in predicting drug responses using omics data, translating these findings into real clinical practice remains a challenging task.Ideally, these models should be trained on in vivo data to ensure clinical relevance and applicability of their predictions. 8However, the omics data related to drug therapeutic responses predominantly come from in vitro studies of cell lines, with a limited availability of omics data obtained from actual clinical patients. 9Therefore, we need a model that not only performs well on cell line data but also demonstrates robust performance when utilized with clinical patient data.
Deep learning, as a powerful machine learning method, 10 has demonstrated tremendous potential in predicting drug treatment responses.However, deep learning models still face challenges when applied in clinical practice.First, deep learning models typically require a large amount of data for training, while clinical patient data is often limited.Second, deep learning models are often considered as 'black boxes' with a lack of interpretability, which is unacceptable in clinical decision-making.Therefore, exploring ways to extend deep learning models to address these issues is an important research direction.In recent years, some researchers have proposed methods to enhance the interpretability of deep learning models.For instance, the introduction of attention mechanisms 11 allows the model to focus on important features relevant to the prediction task, 12 thereby improving interpretability.Additionally, some researchers have attempted to incorporate domain knowledge and prior information into deep learning models to enhance their interpretability and generalization ability.For example, integrating biological pathway information can help explain the biological processes involved in predictions of the model. 13knowledging the common challenges of limited clinical applicability in models based on cell line data and the lack of biological interpretability often observed in deep learning models, we propose an interpretable model called pathway-based attention convolution (PBAC) for the prediction of responses to clinical treatment.Specifically, we introduced a layer of pathway nodes and their connections to input gene nodes, assigning weights to pathway nodes before convolution.This framework presents two distinct advantages: first, it provides interpretability, enabling users to comprehend the crucial pathways involved in delineating a drug's response features.The second benefit is that even when models are trained on baseline data, they demonstrate exceptional performance when tested against clinical data.We conducted a comprehensive evaluation of various drug sensitivity and immunotherapy datasets, includ-

| PBAC: An interpretable pathway-based attention convolution model
We propose a deep learning framework that can be utilized in clinical data for drug therapy response prediction.Specifically, the PBAC consists of four layers: gene-pathway layer, attention layer, convolutional layer and fully connected layer.

| Gene-pathway layer
The connections between the initial input layer and pathway layer are determined by the correlations between genes and pathways.
A mask matrix, represented by M and with dimensions of 1358 × n (where, n represents the number of genes), was utilized to encode the relationship between gene nodes and pathway nodes.In this matrix, a value of 1 indicates the presence of an association between a gene and pathway node, while 0 signifies the lack of such an association.This process can be illustrated as: where X m represents the filtered gene expression data, M is the genepathway connection matrix and X is the original gene expression data.

| Attention layer
In our model, we integrate an attention mechanism to determine and evaluate the importance of different biological pathways.This mechanism initially projects the features of each pathway into a latent space using a dense layer, followed by a tanh activation function.Then, these projected features are re-mapped to the original feature space through another dense layer, producing attention scores for each specific pathway.These scores are then converted into weights using a softmax function, guaranteeing that the total sum of the weights across all pathways equals 1.This can be represented as: where X a represents the output from the attention layer, W 1 and W 2 are the weights of the fully connected layers and W a denotes the computed attention weights.

| Convolutional layer and fully connected layer
We utilize a one-dimensional convolutional layer to process the output from the attention mechanism.The objective of this layer's design is to capture and learn local correlations among different pathway features.Specifically, our convolutional layer is configured with a kernel size of 3, a stride of 1, and padding of 1.The fully connected layer is employed for the final classification task.It transforms the features extracted by the convolutional layer into the target class space, thereby generating the probabilities for each category.Mathematically, the operations of the convolutional layer and fully connected layer can be represented as: where X c is the output from the convolutional layer, Y represents the output probabilities for each category, W f denotes the weights of the fully connected layer and b signifies the bias term.

| Statistical analysis
The effect size was computed as the mean difference in predicted IC50 z-scores of responders versus non-responders.The p-value was calculated using the Wilcoxon rank sum (Mann-Whitney) test.
The analysis was conducted in Python 3.7.For the immune infiltration analysis, we utilized the Cibersort package in R 4.1 and employed the Mann-Whitney-Wilcoxon test.In all statistical analyses, a significance level of p < 0.05 was set.

| RE SULTS
As shown in Figure 1, we trained PBAC on GDSC cell lines screened with chemotherapeutic agents.The input, consisting of gene expression data, undergoes initial processing through a gene-pathway masking layer.This layer functions to eliminate genes not associated with the critical pathways involved in the disease under investigation.Subsequently, an attention mechanism is utilized to prioritize pathway features that significantly impact the predictive outcome.
Following this, convolution operations are applied for robust feature extraction.Ultimately, the model performs a classification prediction, offering valuable insights for disease prognosis or diagnosis.
We assessed the model using clinical patient data related to specific drugs and successfully integrated immunotherapy data into the model.Based on the model's classification results, we conducted immune infiltration analysis.Additionally, using the attention scores from the model, we identified pathways that are highly correlated with disease treatment. 24

| Chemotherapy drug response prediction
In the current study, we trained our model using cell line data associated with four chemotherapy drugs: Bortezomib, Cisplatin, Docetaxel and Paclitaxel.Subsequently, we tested the model using clinical patient data specific to these four drugs.The performance of our model was evaluated based on three key metrics: The AUROC, AUPRC and effect size.We compare the performance of our model with MOLI 8 and the method proposed by Deng et al. 13 Additionally, we contrasted our approach with conventional machine learning methods, namely Random Forest and Support Vector Machine.As

| Ablation study in the PBAC model
In the ablation experiment, we investigated the individual contributions of the masking layer and attention layer to the overall performance of the PBAC model.By systematically removing each of these components from the model and observing the subsequent effect on its predictive performance, we were able to quantify their respective roles in chemotherapy drug response prediction.Figure 4A

| Immunotherapy response prediction
Over the past few years, immunotherapy has significantly revolutionized the clinical treatment landscape for cancer patients. 25Despite the substantial clinical benefits offered by immunotherapy, a notable limitation is that only a minority of patients (~30% in solid tumours) exhibit responsiveness to this treatment.Furthermore, there's a potential for toxicity following immunotherapy. 26Therefore, we used the PBAC model to predict treatment responses in the immunotherapy datasets.Our model was assessed utilizing 11 distinct immunotherapy datasets, comprising over 1000 patient samples across various cancer types and immunotherapy modalities.2) across-study predictions, where two distinct datasets were deployed as the training and test sets. 27 the within-study predictions, we conducted five-fold cross among different groups (Figure 6C).In the GSE91061 dataset, significant distinctions were noted in the proportions of mast cells resting, macrophages M1 and T cells follicular helper, among others (Figure 6D).Our study indicates that, even in the cross-dataset prediction, PBAC retains a consistent level of performance and outperforms alternative methods.

| Biological interpretability of the PBAC model
Advantage of PBAC lies in its interpretability, coupled with its ability to identify significant pathway features for predicting treatment responses.Identifying pathways related to drug response is crucial for comprehending the mechanism of drug action.To achieve this, we ranked the importance of pathways in predicting drug treatment responses based on the attention scores obtained from the model.For instance, in the pathway ranking of the cisplatin drug dataset GSE18864 (Figure 7A), we identified a cytochrome P450 related pathway (i.e.Drug metabolism-cytochrome P450).Cytochrome P450 (CYP450) plays a crucial role in drug metabolism, with studies showing that this pathway is highly associated with anti-breast cancer drugs, particularly serving as an important drug target in breast cancer treatment. 29As the major anti-cancer drug for breast cancer, the drug is metabolized to its active metabolites by the cytochrome P450. 30In the melanoma immunotherapy dataset GSE19293, we identified a cell

FIGURE 5 Legend on next page
apoptosis pathway related to melanoma treatment (Figure 7B).
Dysregulation of the apoptosis pathway hinders the treatment of melanoma. 31Recent research also indicated that apoptosis induced by anticancer drugs may be a predictor of drug sensitivity. 32r example, proteasome inhibitors alone, or in combination with other drugs, efficiently induce apoptosis in melanoma cells and the apoptosis status is the indicator of drug efficiency. 33In the top-ranked pathways identified in the GSE19293 dataset, we also discovered the purine metabolism pathway.The enzyme complex within the purine metabolism pathway could serve as a potential target for tumour treatment. 34Purines are basic components of nucleotides in cell proliferation and the impaired purine metabolism is associated with the progression of cancer.The major anticancer drug-rapamycin is the purinosome formation and purine metabolism regulator, the purine-related metabolism process is served as the target of rapamycin. 35Remarkably, within the top 10 pathways of the GSE106128 dataset (Figure 7C), we also identified the presence of pathway purine metabolism.Due to individual differences, the top 10 pathways obtained in different datasets were also different.However, we still found some relevant pathways in other melanoma datasets.For example, in Liu et al. dataset, we identified the 'Regulation of HSF1-mediated heat shock response' pathway (Figure 7D), which is related to the human melanoma phenotype. 36Besides, the expression of the IL1 pathway, which ranks first in attention scores in the IMvigor210 bladder cancer dataset (Figure 7E), may be associated with the invasive capability of bladder cancer cell lines. 37In the top 10 pathways of the glioblastoma multiforme (GBM) dataset (Figure 7F), the chemokine signalling pathway is associated with tumour progression, and in GBM, the chemokine receptor signalling pathway is activated. 38In summary, PBAC is capable of inferring the importance of specific biological pathways in treatment responses.Furthermore, we utilized an attention mechanism to assign higher weights to pathways that are more important for predicting outcomes. 11,40Through analysing the weights, we can identify pathways that play crucial roles in specific drug treatments.Due to the large number of pathways, the features are also high-dimensional.

| DISCUSS ION
ing cell line data related to four distinct chemotherapeutic agents (Bortezomib, Cisplatin, Docetaxel and Paclitaxel) along with their corresponding clinical datasets, as well as 11 unique datasets associated with immunotherapy.Our model showed superior performance across most of the datasets, outperforming traditional machine learning and deep learning methods.Additionally, through pathway scoring, we identified several biological pathways relevant to disease treatment.Overall, our model provides biological interpretability for predicting drug response and immunotherapy response, effectively bridging the gap between cell line data to clinical data.

For
drug sensitivity data related to cell lines, we classify the cell lines based on the Maximum Screening Concentration (Max Conc) provided in the GDSC database.If the IC50 value of a cell line exceeds the Max Conc, it is labelled as insensitive; otherwise, it is deemed sensitive.The Max Conc values for Bortezomib, Cisplatin, Docetaxel and Paclitaxel shown in Figure2A-H, our model delivers reliable predictions of chemotherapy drug responses, surpassing previous research methods and conventional machine learning approaches.To further demonstrate the predictive capability of the model, we utilized effect size in drug sensitivity prediction.We adjusted the model's output to IC50 values, avoiding the binary classification into 0 and 1 during label processing for the cell line data.Subsequently, we calculated the effect size based on the predicted IC50 values and the response categories of clinical patients.Furthermore, the effect size of the PBAC model was superior to other methods across all four drugs (Figure 3A-D).
show the ROC curves of the PBAC model after removing the gene pathway masking module and attention module.According to the trends of the ROC curves and AUC values, it is evident that the predictive performance of the PBAC model for the efficacy of four chemotherapy drugs significantly decreases in the ablation experiment.Similarly, from the precise recall curves of the PBAC model in the ablation experiment (Figure4E-H), we find that AUPRC also decreases significantly after removing each module individually.This indicates the critical role of the masking layer in managing input variations and reducing the impact of irrelevant features, while the attention layer effectively focuses on key predictive features during the learning process.

| 7 of 14 DENG
validation on each dataset (Figure5A-D), and calculated the average AUC (Figure5E) and AUPRC (Figure5F) to assess the performance of the model.Overall, among the 11 immunotherapy datasets we collected, our model outperformed both Deng et al.13 method and traditional machine learning methods in eight datasets.During the cross-validation process, datasets GSE19293 and GSE106128 demonstrated exceptional performance with AUC values exceeding 0.84.Furthermore, their respective AUPRC reached notable scores of 0.85 and 0.91, respectively.In dataset GSE78220, although the AUC values of PBAC and Deng's method are equal, PBAC demonstrates a higher AUPRC.To show the significant influence of the masking layer and attention layer on predicting immunotherapy response, similar to the process of predicting chemotherapy response, we conducted ablation experiments on these high-performing datasets.The removal of the mask layer and attention layer results in varying degrees of decline in both AUC and AUPRC (Figure5G).For the across-study predictions, we chose to focus on melanoma due to its prevalence in our datasets.The phs000452 dataset, with the largest sample size (153 samples) among the melanoma datasets, was selected as the training set.The remaining seven datasets were used as the test sets.PBAC demonstrated significant superiority over other models in five of these datasets (Figure 6A,B).Our model shows superior performance in datasets with larger sample sizes.In Liu et al. dataset with 121 samples, PBAC performs the best (AUC = 0.80), whereas in datasets F I G U R E 1 The overview of PBAC for drug therapy prediction framework.The gene expression data is processed through a genepathway mask layer, an attention mechanism, and convolution for classification of the prediction results.Furthermore, the model undergoes evaluation and interpretability analysis.F I G U R E 2 The comparison of ROC curves and the precision-recall curves of PBAC model in clinical data with traditional machine learning and existing research methods.(A-D) Are the ROC curves of four chemotherapy drugs.(E-H) Are the precision-recall curve of four chemotherapy drugs.et al.GSE78220 (27 samples) and GSE100797 (25 samples) with sample sizes less than 30, PBAC performs worse than other models.To validate our prediction results, we employed the CIBERSORT algorithm 28 to analyse immune cell types in gene expression data, and integrated the results of CIBERSORT with the group information we predicted.Subsequently, we performed a Wilcoxon test for each type of immune cell between different groups and visualized the results using box plots.In the grouping based on the PBAC model, we observed significant variances in immune cell types between the response group and non-response group.For instance, in Liu et al. dataset, notable differences were found in the proportions of B cells naive, B cells memory and plasma cells

F I G U R E 3 | 9 of 14 DENG
Volcano plots of effect size (difference of mean IC50 between responders and non-responders) and p-values (Wilcox test) of each learning algorithm.F I G U R E 4 The comparison of AUC and Area Under the Precision-Recall Curve (AUPRC) in PBAC's Ablation experiment.(A-D) are the ROC curve of ablation experiments for four different drugs, while (E-H) represent the precision-recall curves of four different drugs.et al.

10 F I G U R E 5 F I G U R E 7
To further extract features, reduce the model's training parameters and prevent overfitting, we incorporated convolutional layers to enhance the model performance.We trained the PBAC model on the datasets of Bortezomib, Cisplatin, Docetaxel and Paclitaxel chemotherapy drugs, and evaluated its performance on corresponding clinical data.Our model significantly outperformed MOLI, 8 the method proposed by Deng et al.,13 as well as traditional machine learning methods such as random forests and support vector machines.To further validate the stability and clinical utility of the model, we assessed the performance of PBAC on 11 immunotherapy datasets, with results showing that our model performed better in most datasets.More importantly, PBAC managed to capture some complex processes of biological interaction based on the attention mechanism.In chemotherapy, we identified the breast cancer-related pathway Cytochrome P450 (CYP450) in the cisplatin treatment data.This pathway plays a crucial role as a drug target in breast cancer treatment,30 indicating that our model can identify important pathways for specific drugs in disease treatment, helping us understand the mechanism of drug action.Additionally, numerous insights were gained in immunotherapy.For example, by ranking pathway attention scores in the melanoma dataset GSE19293, we identified several pathways closely associated with melanoma.Furthermore, in the melanoma dataset GSE106128, we found the same pathways as in GSE19293, validating the stability of our model and demonstrating that PBAC can identify important pathways related to specific diseases.To validate the interpretability of our model in different types of cancer, we generated attention scores for bladder cancer and glioblastoma datasets.As expected, we identified relevant key pathways within their top The ROC curve of the immune therapy datasets in within-study performing five-fold cross-validation on PBAC.(A-D) represent the ROC curves of performance on the GSE19293, GSE106128, IMvigor210 and PRJNA482620, respectively.(E) The AUC performance of PBAC and other methods across eight datasets in within-study.(F) The Area Under the Precision-Recall Curve (AUPRC) performance of PBAC and other methods across eight datasets in within-study.(G) Performance comparison of ablation experiments and PBAC in eight datasets.F I G U R E 6 (A) The comparison of AUC between PBAC and various methods across different datasets in across-study.(B) The comparison of Area Under the Precision-Recall Curve (AUPRC).(C, D) In the dataset Liu et al. and GSE91061, the immune cell components of the two categories were analysed based on the classification results of PBAC (dividing patients into responders (R) and non-responders (NR)).* represents p < 0.05, ** represents p < 0.01, and *** represents p < 0.001.The top 10 pathways ranked by attention scores obtained from training the model on the datasets.The heatmaps for the top 10 pathways of (A-F) correspond respectively to Cisplatin, GSE19293, GSE106128, Liu et al, IMvigor210 and PBJNA482620.
Drug sensitivity data includes cell line and clinical patient data in this study.
23sed in this study, patients with stable disease and progressive disease were classified as non-responders, while those demonstrating either partial response or complete response were identified as responders.We use the Area Under the Receiver Operating Characteristic curve (AUROC) and Area Under the Precision-Recall Curve (AUPRC) to evaluate the predictive performance of the model.Specifically, to further illustrate the predictive performance of the model, we adopted effect size in drug sensitivity prediction.The effect size was calculated by taking the mean difference in the predicted IC50 zscores between responders and non-responders.23 -D

Table 2
provides detailed information.Specifically, we executed two types of predictions: (1) within-study predictions, where both training and test datasets were sourced from a single cohort and (