A Hitchhiker's Guide to Deep Chemical Language Processing for Bioactivity Prediction

Overview

Journal Digit Discov

Publisher Royal Society of Chemistry

Date 2024 Dec 27

PMID 39726698

Authors

Riza Ozcelik

Francesca Grisoni

Affiliations

Soon will be listed here.

Abstract

Deep learning has significantly accelerated drug discovery, with 'chemical language' processing (CLP) emerging as a prominent approach. CLP approaches learn from molecular string representations (, Simplified Molecular Input Line Entry Systems [SMILES] and Self-Referencing Embedded Strings [SELFIES]) with methods akin to natural language processing. Despite their growing importance, training predictive CLP models is far from trivial, as it involves many 'bells and whistles'. Here, we analyze the key elements of CLP and provide guidelines for newcomers and experts. Our study spans three neural network architectures, two string representations, three embedding strategies, across ten bioactivity datasets, for both classification and regression purposes. This 'hitchhiker's guide' not only underscores the importance of certain methodological decisions, but it also equips researchers with practical recommendations on ideal choices, , in terms of neural network architectures, molecular representations, and hyperparameter optimization.

References

Ozturk H, Ozgur A, Ozkirimli E . DeepDTA: deep drug-target binding affinity prediction. Bioinformatics. 2018; 34(17):i821-i829. PMC: 6129291. DOI: 10.1093/bioinformatics/bty593. View

Krenn M, Ai Q, Barthel S, Carson N, Frei A, Frey N . SELFIES and the future of molecular string representations. Patterns (N Y). 2022; 3(10):100588. PMC: 9583042. DOI: 10.1016/j.patter.2022.100588. View

Ozcelik R, Ozturk H, Ozgur A, Ozkirimli E . ChemBoost: A Chemical Language Based Approach for Protein - Ligand Binding Affinity Prediction. Mol Inform. 2020; 40(5):e2000212. DOI: 10.1002/minf.202000212. View

Cai C, Wang S, Xu Y, Zhang W, Tang K, Ouyang Q . Transfer Learning for Drug Discovery. J Med Chem. 2020; 63(16):8683-8694. DOI: 10.1021/acs.jmedchem.9b02147. View

Zhao Q, Duan G, Yang M, Cheng Z, Li Y, Wang J . AttentionDTA: Drug-Target Binding Affinity Prediction by Sequence-Based Deep Learning With Attention Mechanism. IEEE/ACM Trans Comput Biol Bioinform. 2022; 20(2):852-863. DOI: 10.1109/TCBB.2022.3170365. View

Flam-Shepherd D, Zhu K, Aspuru-Guzik A . Language models can learn complex molecular distributions. Nat Commun. 2022; 13(1):3293. PMC: 9174447. DOI: 10.1038/s41467-022-30839-x. View

Shemetulskis N, Weininger D, Blankley C, Yang J, Humblet C . Stigmata: an algorithm to determine structural commonalities in diverse datasets. J Chem Inf Comput Sci. 1996; 36(4):862-71. DOI: 10.1021/ci950169+. View

van Tilborg D, Alenicheva A, Grisoni F . Exposing the Limitations of Molecular Machine Learning with Activity Cliffs. J Chem Inf Model. 2022; 62(23):5938-5951. PMC: 9749029. DOI: 10.1021/acs.jcim.2c01073. View

Zeng X, Li S, Lv S, Wen M, Li Y . A comprehensive review of the recent advances on predicting drug-target affinity based on deep learning. Front Pharmacol. 2024; 15:1375522. PMC: 11019008. DOI: 10.3389/fphar.2024.1375522. View

10.

Hopfield J . Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A. 1982; 79(8):2554-8. PMC: 346238. DOI: 10.1073/pnas.79.8.2554. View

11.

Grisoni F . Chemical language models for de novo drug design: Challenges and opportunities. Curr Opin Struct Biol. 2023; 79:102527. DOI: 10.1016/j.sbi.2023.102527. View

12.

Ozturk H, Ozgur A, Schwaller P, Laino T, Ozkirimli E . Exploring chemical space using natural language processing methodologies for drug discovery. Drug Discov Today. 2020; 25(4):689-705. DOI: 10.1016/j.drudis.2020.01.020. View

13.

Uludogan G, Ozkirimli E, Ulgen K, Karali N, Ozgur A . Exploiting pretrained biochemical language models for targeted drug design. Bioinformatics. 2022; 38(Suppl_2):ii155-ii161. DOI: 10.1093/bioinformatics/btac482. View

14.

Wu J, Wang T, Chen Y, Tang L, Wu H, Yu R . t-SMILES: a fragment-based molecular representation framework for de novo ligand design. Nat Commun. 2024; 15(1):4993. PMC: 11167009. DOI: 10.1038/s41467-024-49388-6. View

15.

Wieder O, Kohlbacher S, Kuenemann M, Garon A, Ducrot P, Seidel T . A compact review of molecular property prediction with graph neural networks. Drug Discov Today Technol. 2021; 37:1-12. DOI: 10.1016/j.ddtec.2020.11.009. View

16.

Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G . Applications of machine learning in drug discovery and development. Nat Rev Drug Discov. 2019; 18(6):463-477. PMC: 6552674. DOI: 10.1038/s41573-019-0024-5. View

17.

Sharma A, Kumar R, Ranjta S, Varadwaj P . SMILES to Smell: Decoding the Structure-Odor Relationship of Chemical Compounds Using the Deep Neural Network Approach. J Chem Inf Model. 2021; 61(2):676-688. DOI: 10.1021/acs.jcim.0c01288. View

18.

Wu C, Zhang X, Yang Z, Lu A, Hou T, Cao D . Learning to SMILES: BAN-based strategies to improve latent representation learning from molecules. Brief Bioinform. 2021; 22(6). DOI: 10.1093/bib/bbab327. View

19.

Sun J, Jeliazkova N, Chupakhin V, Golib-Dzib J, Engkvist O, Carlsson L . Erratum to: ExCAPE-DB: an integrated large scale dataset facilitating Big Data analysis in chemogenomics. J Cheminform. 2017; 9(1):41. PMC: 5471272. DOI: 10.1186/s13321-017-0222-2. View

20.

Zhou Y, Cahya S, Combs S, Nicolaou C, Wang J, Desai P . Exploring Tunable Hyperparameters for Deep Neural Networks with Industrial ADME Data Sets. J Chem Inf Model. 2018; 59(3):1005-1016. DOI: 10.1021/acs.jcim.8b00671. View