Hybrid Fragment-SMILES Tokenization for ADMET Prediction in Drug Discovery

Overview

Journal BMC Bioinformatics

Publisher Biomed Central

Specialty Biology

Date 2024 Aug 1

PMID 39090573

Authors

Nicholas Aksamit

Alain Tchagang

Yifeng Li

Beatrice Ombuki-Berman

Affiliations

Soon will be listed here.

Abstract

Background: Drug discovery and development is the extremely costly and time-consuming process of identifying new molecules that can interact with a biomarker target to interrupt the disease pathway of interest. In addition to binding the target, a drug candidate needs to satisfy multiple properties affecting absorption, distribution, metabolism, excretion, and toxicity (ADMET). Artificial intelligence approaches provide an opportunity to improve each step of the drug discovery and development process, in which the first question faced by us is how a molecule can be informatively represented such that the in-silico solutions are optimized.

Results: This study introduces a novel hybrid SMILES-fragment tokenization method, coupled with two pre-training strategies, utilizing a Transformer-based model. We investigate the efficacy of hybrid tokenization in improving the performance of ADMET prediction tasks. Our approach leverages MTL-BERT, an encoder-only Transformer model that achieves state-of-the-art ADMET predictions, and contrasts the standard SMILES tokenization with our hybrid method across a spectrum of fragment library cutoffs.

Conclusion: The findings reveal that while an excess of fragments can impede performance, using hybrid tokenization with high frequency fragments enhances results beyond the base SMILES tokenization. This advancement underscores the potential of integrating fragment- and character-level molecular features within the training of Transformer models for ADMET property prediction.

References

Tran T, Wibowo A, Tayara H, Chong K . Artificial Intelligence in Drug Toxicity Prediction: Recent Advances, Challenges, and Future Perspectives. J Chem Inf Model. 2023; 63(9):2628-2643. DOI: 10.1021/acs.jcim.3c00200. View

Montanari F, Kuhnke L, Ter Laak A, Clevert D . Modeling Physico-Chemical ADMET Endpoints with Multitask Graph Convolutional Networks. Molecules. 2019; 25(1). PMC: 6982787. DOI: 10.3390/molecules25010044. View

Song Y, Chen J, Wang W, Chen G, Ma Z . Double-head transformer neural network for molecular property prediction. J Cheminform. 2023; 15(1):27. PMC: 9951429. DOI: 10.1186/s13321-023-00700-4. View

Zhang S, Yan Z, Huang Y, Liu L, He D, Wang W . HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer. Bioinformatics. 2022; 38(13):3444-3453. DOI: 10.1093/bioinformatics/btac342. View

Heid E, Greenman K, Chung Y, Li S, Graff D, Vermeire F . Chemprop: A Machine Learning Package for Chemical Property Prediction. J Chem Inf Model. 2023; 64(1):9-17. PMC: 10777403. DOI: 10.1021/acs.jcim.3c01250. View

Wei Y, Li S, Li Z, Wan Z, Lin J . Interpretable-ADMET: a web service for ADMET prediction and optimization based on deep neural representation. Bioinformatics. 2022; 38(10):2863-2871. DOI: 10.1093/bioinformatics/btac192. View

Yang L, Jin C, Yang G, Bing Z, Huang L, Niu Y . Transformer-based deep learning method for optimizing ADMET properties of lead compounds. Phys Chem Chem Phys. 2023; 25(3):2377-2385. DOI: 10.1039/d2cp05332b. View

Wu Z, Ramsundar B, Feinberg E, Gomes J, Geniesse C, Pappu A . MoleculeNet: a benchmark for molecular machine learning. Chem Sci. 2018; 9(2):513-530. PMC: 5868307. DOI: 10.1039/c7sc02664a. View

Wu Z, Jiang D, Wang J, Zhang X, Du H, Pan L . Knowledge-based BERT: a method to extract molecular features like computational chemists. Brief Bioinform. 2022; 23(3). DOI: 10.1093/bib/bbac131. View

10.

Coley C, Barzilay R, Green W, Jaakkola T, Jensen K . Convolutional Embedding of Attributed Molecular Graphs for Physical Property Prediction. J Chem Inf Model. 2017; 57(8):1757-1772. DOI: 10.1021/acs.jcim.6b00601. View

11.

Xiong G, Wu Z, Yi J, Fu L, Yang Z, Hsieh C . ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET properties. Nucleic Acids Res. 2021; 49(W1):W5-W14. PMC: 8262709. DOI: 10.1093/nar/gkab255. View

12.

Liu Y, Zhang R, Li T, Jiang J, Ma J, Wang P . MolRoPE-BERT: An enhanced molecular representation with Rotary Position Embedding for molecular property prediction. J Mol Graph Model. 2022; 118:108344. DOI: 10.1016/j.jmgm.2022.108344. View

13.

Polykovskiy D, Zhebrak A, Sanchez-Lengeling B, Golovanov S, Tatanov O, Belyaev S . Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models. Front Pharmacol. 2021; 11:565644. PMC: 7775580. DOI: 10.3389/fphar.2020.565644. View

14.

Kumar M, Nguyen T, Kaur J, Singh T, Soni D, Singh R . Opportunities and challenges in application of artificial intelligence in pharmacology. Pharmacol Rep. 2023; 75(1):3-18. PMC: 9838466. DOI: 10.1007/s43440-022-00445-1. View

15.

Wieder O, Kohlbacher S, Kuenemann M, Garon A, Ducrot P, Seidel T . A compact review of molecular property prediction with graph neural networks. Drug Discov Today Technol. 2021; 37:1-12. DOI: 10.1016/j.ddtec.2020.11.009. View

16.

Lipinski C, Maltarollo V, Oliveira P, da Silva A, Honorio K . Advances and Perspectives in Applying Deep Learning for Drug Design and Discovery. Front Robot AI. 2021; 6:108. PMC: 7805776. DOI: 10.3389/frobt.2019.00108. View

17.

Stahl N, Falkman G, Karlsson A, Mathiason G, Bostrom J . Deep Reinforcement Learning for Multiparameter Optimization in Drug Design. J Chem Inf Model. 2019; 59(7):3166-3176. DOI: 10.1021/acs.jcim.9b00325. View

18.

Ferreira L, Andricopulo A . ADMET modeling approaches in drug discovery. Drug Discov Today. 2019; 24(5):1157-1165. DOI: 10.1016/j.drudis.2019.03.015. View

19.

Zhang X, Wu C, Yang Z, Wu Z, Yi J, Hsieh C . MG-BERT: leveraging unsupervised atomic representation learning for molecular property prediction. Brief Bioinform. 2021; 22(6). DOI: 10.1093/bib/bbab152. View

20.

Feinberg E, Joshi E, Pande V, Cheng A . Improvement in ADMET Prediction with Multitask Deep Featurization. J Med Chem. 2020; 63(16):8835-8848. DOI: 10.1021/acs.jmedchem.9b02187. View