Direct Coupling Analysis and the Attention Mechanism

Overview

Journal BMC Bioinformatics

Publisher Biomed Central

Date 2025 Feb 6

PMID 39915710

Authors

Francesco Caredda

Andrea Pagnani

Affiliations

Soon will be listed here.

Abstract

Proteins are involved in nearly all cellular functions, encompassing roles in transport, signaling, enzymatic activity, and more. Their functionalities crucially depend on their complex three-dimensional arrangement. For this reason, being able to predict their structure from the amino acid sequence has been and still is a phenomenal computational challenge that the introduction of AlphaFold solved with unprecedented accuracy. However, the inherent complexity of AlphaFold's architectures makes it challenging to understand the rules that ultimately shape the protein's predicted structure. This study investigates a single-layer unsupervised model based on the attention mechanism. More precisely, we explore a Direct Coupling Analysis (DCA) method that mimics the attention mechanism of several popular Transformer architectures, such as AlphaFold itself. The model's parameters, notably fewer than those in standard DCA-based algorithms, can be directly used for extracting structural determinants such as the contact map of the protein family under study. Additionally, the functional form of the energy function of the model enables us to deploy a multi-family learning strategy, allowing us to effectively integrate information across multiple protein families, whereas standard DCA algorithms are typically limited to single protein families. Finally, we implemented a generative version of the model using an autoregressive architecture, capable of efficiently generating new proteins in silico.

References

Gueudre T, Baldassi C, Zamparo M, Weigt M, Pagnani A . Simultaneous identification of specifically interacting paralogs and interprotein contacts by direct coupling analysis. Proc Natl Acad Sci U S A. 2016; 113(43):12186-12191. PMC: 5087065. DOI: 10.1073/pnas.1607570113. View

Morcos F, Pagnani A, Lunt B, Bertolino A, Marks D, Sander C . Direct-coupling analysis of residue coevolution captures native contacts across many protein families. Proc Natl Acad Sci U S A. 2011; 108(49):E1293-301. PMC: 3241805. DOI: 10.1073/pnas.1111471108. View

Fernandez-de-Cossio-Diaz J, Uguzzoni G, Pagnani A . Unsupervised Inference of Protein Fitness Landscape from Deep Mutational Scan. Mol Biol Evol. 2020; 38(1):318-328. PMC: 7783173. DOI: 10.1093/molbev/msaa204. View

Figliuzzi M, Barrat-Charlaix P, Weigt M . How Pairwise Coevolutionary Models Capture the Collective Residue Variability in Proteins?. Mol Biol Evol. 2018; 35(4):1018-1027. DOI: 10.1093/molbev/msy007. View

Heinzinger M, Elnaggar A, Wang Y, Dallago C, Nechaev D, Matthes F . Modeling aspects of the language of life through transfer-learning protein sequences. BMC Bioinformatics. 2019; 20(1):723. PMC: 6918593. DOI: 10.1186/s12859-019-3220-8. View

Ekeberg M, Lovkvist C, Lan Y, Weigt M, Aurell E . Improved contact prediction in proteins: using pseudolikelihoods to infer Potts models. Phys Rev E Stat Nonlin Soft Matter Phys. 2013; 87(1):012707. DOI: 10.1103/PhysRevE.87.012707. View

Hopf T, Colwell L, Sheridan R, Rost B, Sander C, Marks D . Three-dimensional structures of membrane proteins from genomic sequencing. Cell. 2012; 149(7):1607-21. PMC: 3641781. DOI: 10.1016/j.cell.2012.04.012. View

Sesta L, Pagnani A, Fernandez-de-Cossio-Diaz J, Uguzzoni G . Inference of annealed protein fitness landscapes with AnnealDCA. PLoS Comput Biol. 2024; 20(2):e1011812. PMC: 10878520. DOI: 10.1371/journal.pcbi.1011812. View

Kuhlman B, Dantas G, Ireton G, Varani G, Stoddard B, Baker D . Design of a novel globular protein fold with atomic-level accuracy. Science. 2003; 302(5649):1364-8. DOI: 10.1126/science.1089427. View

10.

Dill K, MacCallum J . The protein-folding problem, 50 years on. Science. 2012; 338(6110):1042-6. DOI: 10.1126/science.1219021. View

11.

Cocco S, Monasson R, Sessak V . High-dimensional inference with the generalized Hopfield model: principal component analysis and corrections. Phys Rev E Stat Nonlin Soft Matter Phys. 2011; 83(5 Pt 1):051123. DOI: 10.1103/PhysRevE.83.051123. View

12.

Hopf T, Ingraham J, Poelwijk F, Scharfe C, Springer M, Sander C . Mutation effects predicted from sequence co-variation. Nat Biotechnol. 2017; 35(2):128-135. PMC: 5383098. DOI: 10.1038/nbt.3769. View

13.

Shakhnovich E, Gutin A . Engineering of stable and fast-folding sequences of model proteins. Proc Natl Acad Sci U S A. 1993; 90(15):7195-9. PMC: 47103. DOI: 10.1073/pnas.90.15.7195. View

14.

Dunn S, Wahl L, Gloor G . Mutual information without the influence of phylogeny or entropy dramatically improves residue contact prediction. Bioinformatics. 2007; 24(3):333-40. DOI: 10.1093/bioinformatics/btm604. View

15.

Bitbol A, Dwyer R, Colwell L, Wingreen N . Inferring interaction partners from protein sequences. Proc Natl Acad Sci U S A. 2016; 113(43):12180-12185. PMC: 5087060. DOI: 10.1073/pnas.1606762113. View

16.

Weigt M, White R, Szurmant H, Hoch J, Hwa T . Identification of direct residue contacts in protein-protein interaction by message passing. Proc Natl Acad Sci U S A. 2009; 106(1):67-72. PMC: 2629192. DOI: 10.1073/pnas.0805923106. View

17.

Barrat-Charlaix P, Muntoni A, Shimagaki K, Weigt M, Zamponi F . Sparse generative modeling via parameter reduction of Boltzmann machines: Application to protein-sequence families. Phys Rev E. 2021; 104(2-1):024407. DOI: 10.1103/PhysRevE.104.024407. View

18.

Berman H, Westbrook J, Feng Z, Gilliland G, Bhat T, Weissig H . The Protein Data Bank. Nucleic Acids Res. 1999; 28(1):235-42. PMC: 102472. DOI: 10.1093/nar/28.1.235. View

19.

Morcos F, Jana B, Hwa T, Onuchic J . Coevolutionary signals across protein lineages help capture multiple protein conformations. Proc Natl Acad Sci U S A. 2013; 110(51):20533-8. PMC: 3870752. DOI: 10.1073/pnas.1315625110. View

20.

Taylor W, Sadowski M . Structural constraints on the covariance matrix derived from multiple aligned protein sequences. PLoS One. 2011; 6(12):e28265. PMC: 3237328. DOI: 10.1371/journal.pone.0028265. View