DTIBFAI: drug-target interaction prediction based on BERT and feature augment of Informer

Abstract

Drug-target interaction (DTI) is critical for drug discovery, providing insights into novel therapies. The development of language models has provided strong support for DTI prediction. However, Transformer-based self-attention mechanisms often fail to capture fine-grained drug-target interactions, and single-mode feature representations limit the ability to fully characterize DTI. To address these issues, this study proposes a novel prediction method DTIBFAI based on BERT and Informer model. DTIBFAI preprocesses drug and protein sequences using ChemBERTa and BioBERT, while incorporating molecular fingerprints and dipeptide composition features to augment the richness and representativeness of the features. Additionally, this study integrates a modified Informer for the DTI prediction problem. The modified Informer augments feature embeddings and effectively captures the complex interaction patterns within sequence data. The performance of the DTIBFAI model is evaluated by comparing it with several state-of-the-art methods for drug-target interaction prediction. Experimental results demonstrate that DTIBFAI significantly outperforms these methods on the evaluated datasets, achieving AUROC and AUPRC scores of 0.9661 and 0.9673, respectively. Case studies reveal the model’s ability to identify novel DTI, including previously unrecorded interactions, validated for their biological plausibility. These results demonstrate the potential of DTIBFAI in advancing DTI prediction and its application in drug discovery. The code and data of DTIBFAI are available at the website of github.com/KNDF001/DTIBFAI-Drug-target-Interaction-Prediction-Based-on-BERT-and-Feature-Augment-of-Informer.

Graphical abstract

Keywords

DTI prediction / BioBERT / ChemBERTa / multi-feature fusion / Informer

References

[1] Kapetanovic I M . Computer-aided drug discovery and development (CADDD): in silico-chemico-biological approach. Chemico-Biological Interactions, 2008, 171( 2): 165–176

[2] Dara S, Dhamercherla S, Jadav S S, Babu C M, Ahsan M J . Machine learning in drug discovery: a review. Artificial Intelligence Review, 2022, 55( 3): 1947–1999

[3] Chu Z, Huang F, Fu H, Quan Y, Zhou X, Liu S, Zhang W . Hierarchical graph representation learning for the prediction of drug-target binding affinity. Information Sciences, 2022, 613: 507–523

[4] Roy R, Al-Hashimi H M . AlphaFold3 takes a step toward decoding molecular behavior and biological computation. Nature Structural & Molecular Biology, 2024, 31( 7): 997–1000

[5] Chen L, Tan X, Wang D, Zhong F, Liu X, Yang T, Luo X, Chen K, Jiang H, Zheng M . TransformerCPI: improving compound-protein interaction prediction by sequence-based deep learning with self-attention mechanism and label reversal experiments. Bioinformatics, 2020, 36( 16): 4406–4414

[6] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010

[7] Zhao Q, Duan G, Yang M, Cheng Z, Li Y, Wang J . AttentionDTA: drug-target binding affinity prediction by sequence-based deep learning with attention mechanism. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2023, 20( 2): 852–863

[8] Kalakoti Y, Yadav S, Sundar D . TransDTI: transformer-based language models for estimating DTIs and building a drug recommendation workflow. ACS Omega, 2022, 7( 3): 2706–2717

[9] Huang K, Fu T, Glass L M, Zitnik M, Xiao C, Sun J. DeepPurpose: a deep learning library for drug-target interaction prediction. Bioinformatics, 2021, 36(22−23): 5545−5547

[10] Karimi M, Wu D, Wang Z, Shen Y . DeepAffinity: interpretable deep learning of compound-protein affinity through unified recurrent and convolutional neural networks. Bioinformatics, 2019, 35( 18): 3329–3338

[11] Wang Z, Chang S, Yang Y, Liu D, Huang T S. Studying very low resolution recognition using deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4792−4800

[12] Weininger D . Smiles. 3. Depict. Graphical depiction of chemical structures. Journal of Chemical Information & Computer Sciences, 1990, 30( 3): 237–243

[13] D’Souza S, Prema K V, Balaji S, Shah R . Deep learning-based modeling of drug-target interaction prediction incorporating binding site information of proteins. Interdisciplinary Sciences: Computational Life Sciences, 2023, 15( 2): 306–315

[14] Wang S, Liu Y, Zhang Y, Zhang K, Song X, Zhang Y, Pang S . CHL-DTI: a novel high-low order information convergence framework for effective drug-target interaction prediction. Interdisciplinary Sciences: Computational Life Sciences, 2024, 16( 3): 568–578

[15] Wang M, Lei X, Liu L, Chen J, Wu F X. GIAE-DTI: predicting drug-target interactions based on heterogeneous network and GIN-based graph autoencoder. IEEE Journal of Biomedical and Health Informatics, 2024: 1−14 (Early Access)

[16] Chen M, Jiang Y, Lei X, Pan Y, Ji C, Jiang W . Drug-target interactions prediction based on signed heterogeneous graph neural networks. Chinese Journal of Electronics, 2024, 33( 1): 231–244

[17] Zhang X, Lei X . Predicting miRNA-drug interactions via dual-channel network based on TCN and BiLSTM. Frontiers of Computer Science, 2025, 19( 5): 195905

[18] Devlin J, Chang M W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186

[19] Chithrananda S, Grand G, Ramsundar B. ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020

[20] Lee J, Yoon W, Kim S, Kim D, Kim S, So C H, Kang J . BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2020, 36( 4): 1234–1240

[21] Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W. Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 11106−11115

[22] Zhang Z, Chen Y, Zhang D, Qian Y, Wang H . CTFNet: long-sequence time-series forecasting based on convolution and time-frequency analysis. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35( 11): 16368–16382

[23] Zhang D, Zhang Z, Chen N, Wang Y . RFNet: multivariate long sequence time-series forecasting based on recurrent representation and feature enhancement. Neural Networks, 2025, 181: 106800

[24] Krizhevsky A, Sutskever I, Hinton G E . ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60( 6): 84–90

[25] Leskovec J, Sosič R. SNAP: A General Purpose Network Analysis and Graph Mining Library. ACM Transactions on Intelligent Systems and Technology, 2016, 8(1): 1

[26] Wishart D S, Knox C, Guo A C, Cheng D, Shrivastava S, Tzur D, Gautam B, Hassanali M . DrugBank: a knowledgebase for drugs, drug actions and drug targets. Nucleic Acids Research, 2008, 36( S1): D901–D906

[27] Wang H, Huang F, Xiong Z, Zhang W . A heterogeneous network-based method with attentive meta-path extraction for predicting drug–target interactions. Briefings in Bioinformatics, 2022, 23( 4): bbac184

[28] Huang Y A, You Z H, Chen X . A systematic prediction of drug-target interactions using molecular fingerprints and protein sequences. Current Protein & Peptide Science, 2018, 19( 5): 468–478

[29] Li J, Sun L, Liu L, Li Z . MIFAM-DTI: a drug-target interactions predicting model based on multi-source information fusion and attention mechanism. Frontiers in Genetics, 2024, 15: 1381997

[30] Rogers D, Hahn M . Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 2010, 50( 5): 742–754

[31] Lovrić M, Molero J M, Kern R . PySpark and RDKit: moving towards big data in cheminformatics. Molecular Informatics, 2019, 38( 6): 1800082

[32] Kim S, Chen J, Cheng T, Gindulyte A, He J, He S, Li Q, Shoemaker B A, Thiessen P A, Yu B, Zaslavsky L, Zhang J, Bolton E E . PubChem 2019 update: improved access to chemical data. Nucleic Acids Research, 2019, 47( D1): D1102–D1109

[33] Sennrich R, Haddow B, Birch A. Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1715–1725

[34] Wu Y, Schuster M, Chen Z, Le Q V, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, Klingner J, Shah A, Johnson M, Liu X, Kaiser Ł, Gouws S, Kato Y, Kudo T, Kazawa H, Stevens K, Kurian G, Patil N, Wang W, Young C, Smith J R, Riesa J, Rudnick A, Vinyals O, Corrado G S, Hughes M, Dean J. Google’s neural machine translation system: bridging the gap between human and machine translation. 2016, arXiv preprint arXiv: 1609.08144

[35] Eckart C, Young G . The approximation of one matrix by another of lower rank. Psychometrika, 1936, 1( 3): 211–218

[36] Clevert D A, Unterthiner T, Hochreiter S. Fast and accurate deep network learning by Exponential Linear Units (ELUs). In: Proceedings of the 4th International Conference on Learning Representations. 2016

[37] Tsubaki M, Tomii K, Sese J . Compound‐protein interaction prediction with end‐to‐end learning of neural networks for graphs and sequences. Bioinformatics, 2019, 35( 2): 309–318

[38] Yin Z, Chen Y, Hao Y, Pandiyan S, Shao J, Wang L . FOTF-CPI: a compound-protein interaction prediction transformer based on the fusion of optimal transport fragments. iScience, 2024, 27( 1): 108756

[39] Peng L, Liu X, Yang L, Liu L, Bai Z, Chen M, Lu X, Nie L . BINDTI: a bi-directional Intention network for drug-target interaction identification based on attention mechanisms. IEEE Journal of Biomedical and Health Informatics, 2025, 29( 3): 1602–1612

[40] Kearnes S, McCloskey K, Berndl M, Pande V, Riley P . Molecular graph convolutions: moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 2016, 30( 8): 595–608

[41] Scarselli F, Gori M, Tsoi A C, Hagenbuchner M, Monfardini G . The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20( 1): 61–80

[42] Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014, 1746−1751

[43] Dauphin Y N, Fan A, Auli M, Grangier D. Language modeling with gated convolutional networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 933–941

[44] Pan X, Ge C, Lu R, Song S, Chen G, Huang Z, Huang G. On the integration of self-attention and convolution. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022, 805−815

[45] Trott O, Olson A J . AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of Computational Chemistry, 2010, 31( 2): 455–461

[46] Markus A Lill, Matthew L Danielson. Computer-aided drug design platform using PyMOL. Journal of Computer-Aided Molecular Design, 2011, 25(1): 13−19

[47] Seeliger D, de Groot B L . Ligand docking and binding site analysis with PyMOL and Autodock/Vina. Journal of Computer-Aided Molecular Design, 2010, 24( 5): 417–422

[48] Consortium U . UniProt: the universal protein knowledgebase in 2025. Nucleic Acids Research, 2025, 53( D1): D609–D617

[49] Kanehisa M, Furumichi M, Sato Y, Matsuura Y, Ishiguro-Watanabe M . KEGG: biological systems database as a model of the real world. Nucleic Acids Research, 2025, 53( D1): D672–D677

[50] Amberger J S, Hamosh A . Searching Online Mendelian Inheritance in Man (OMIM): a knowledgebase of human genes and genetic phenotypes. Current Protocols in Bioinformatics, 2017, 58( 1): 1.2.1–1.2.12

[51] Medioni J, Deplanque G, Ferrero J M, Maurina T, Rodier J M P, Raymond E, Allyon J, Maruani G, Houillier P, Mackenzie S, Renaux S, Dufour-Lamartinie J F, Elaidi R, Lerest C, Oudard S . Phase I safety and pharmacodynamic of inecalcitol, a novel VDR agonist with docetaxel in metastatic castration-resistant prostate cancer patients. Clinical Cancer Research, 2014, 20( 17): 4471–4477

[52] Fu H, Huang F, Liu X, Qiu Y, Zhang W . MVGCN: data integration through multi-view graph convolutional network for predicting links in biomedical bipartite networks. Bioinformatics, 2022, 38( 2): 426–434

RIGHTS & PERMISSIONS

Higher Education Press