PDBench: Evaluating Computational Methods for Protein-sequence Design

Overview

Journal Bioinformatics

Publisher Oxford University Press

Specialty Biology

Date 2023 Jan 13

PMID 36637198

Authors

Leonardo V Castorina

Rokas Petrenas

Kartic Subr

Christopher W Wood

Affiliations

Soon will be listed here.

Abstract

Summary: Ever increasing amounts of protein structure data, combined with advances in machine learning, have led to the rapid proliferation of methods available for protein-sequence design. In order to utilize a design method effectively, it is important to understand the nuances of its performance and how it varies by design target. Here, we present PDBench, a set of proteins and a number of standard tests for assessing the performance of sequence-design methods. PDBench aims to maximize the structural diversity of the benchmark, compared with previous benchmarking sets, in order to provide useful biological insight into the behaviour of sequence-design methods, which is essential for evaluating their performance and practical utility. We believe that these tools are useful for guiding the development of novel sequence design algorithms and will enable users to choose a method that best suits their design target.

Availability And Implementation: https://github.com/wells-wood-research/PDBench.

Supplementary Information: Supplementary data are available at Bioinformatics online.

Citing Articles

SPDesign: protein sequence designer based on structural sequence profile using ultrafast shape recognition.

Wang H, Liu D, Zhao K, Wang Y, Zhang G Brief Bioinform. 2024; 25(3).

PMID: 38600663 PMC: 11006797. DOI: 10.1093/bib/bbae146.

What does it take for an 'AlphaFold Moment' in functional protein engineering and design?.

Chica R, Ferruz N Nat Biotechnol. 2024; 42(2):173-174.

PMID: 38361055 DOI: 10.1038/s41587-023-02120-z.

The blobulator: a webtool for identification and visual exploration of hydrophobic modularity in protein sequences.

Pitman C, Santiago-McRae E, Lohia R, Bassi K, Joseph T, Hansen M bioRxiv. 2024; .

PMID: 38293114 PMC: 10827107. DOI: 10.1101/2024.01.15.575761.

TIMED-Design: flexible and accessible protein sequence design with convolutional neural networks.

Castorina L, Unal S, Subr K, Wood C Protein Eng Des Sel. 2024; 37.

PMID: 38288671 PMC: 10939383. DOI: 10.1093/protein/gzae002.

Multi-indicator comparative evaluation for deep learning-based protein sequence design methods.

Yu J, Mu J, Wei T, Chen H Bioinformatics. 2024; 40(2).

PMID: 38261649 PMC: 10868333. DOI: 10.1093/bioinformatics/btae037.

References

Zhang Y, Chen Y, Wang C, Lo C, Liu X, Wu W . ProDCoNN: Protein design using a convolutional neural network. Proteins. 2019; 88(7):819-829. PMC: 8204568. DOI: 10.1002/prot.25868. View

Ollikainen N, Kortemme T . Computational protein design quantifies structural constraints on amino acid covariation. PLoS Comput Biol. 2013; 9(11):e1003313. PMC: 3828131. DOI: 10.1371/journal.pcbi.1003313. View

Woolfson D, Bartlett G, Burton A, Heal J, Niitsu A, Thomson A . De novo protein design: how do we expand into the universe of possible protein structures?. Curr Opin Struct Biol. 2015; 33:16-26. DOI: 10.1016/j.sbi.2015.05.009. View

Das R, Baker D . Macromolecular modeling with rosetta. Annu Rev Biochem. 2008; 77:363-82. DOI: 10.1146/annurev.biochem.77.062906.171838. View

Chowdhury R, Bouatta N, Biswas S, Floristean C, Kharkar A, Roy K . Single-sequence protein structure prediction using a language model and deep learning. Nat Biotechnol. 2022; 40(11):1617-1623. PMC: 10440047. DOI: 10.1038/s41587-022-01432-w. View

Strokach A, Becerra D, Corbi-Verge C, Perez-Riba A, Kim P . Fast and Flexible Protein Design Using Deep Graph Neural Networks. Cell Syst. 2020; 11(4):402-411.e4. DOI: 10.1016/j.cels.2020.08.016. View

Rives A, Meier J, Sercu T, Goyal S, Lin Z, Liu J . Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc Natl Acad Sci U S A. 2021; 118(15). PMC: 8053943. DOI: 10.1073/pnas.2016239118. View

Qi Y, Zhang J . DenseCPD: Improving the Accuracy of Neural-Network-Based Computational Protein Sequence Design with DenseNet. J Chem Inf Model. 2020; 60(3):1245-1252. DOI: 10.1021/acs.jcim.0c00043. View

Woolfson D . A Brief History of De Novo Protein Design: Minimal, Rational, and Computational. J Mol Biol. 2021; 433(20):167160. DOI: 10.1016/j.jmb.2021.167160. View

10.

OConnell J, Li Z, Hanson J, Heffernan R, Lyons J, Paliwal K . SPIN2: Predicting sequence profiles from protein structures using deep neural networks. Proteins. 2018; 86(6):629-633. DOI: 10.1002/prot.25489. View

11.

Stam M, Wood C . DE-STRESS: a user-friendly web application for the evaluation of protein designs. Protein Eng Des Sel. 2021; 34. PMC: 8672653. DOI: 10.1093/protein/gzab029. View

12.

Huang P, Boyken S, Baker D . The coming of age of de novo protein design. Nature. 2016; 537(7620):320-7. DOI: 10.1038/nature19946. View

13.

Henikoff S, Henikoff J . Amino acid substitution matrices from protein blocks. Proc Natl Acad Sci U S A. 1992; 89(22):10915-9. PMC: 50453. DOI: 10.1073/pnas.89.22.10915. View

14.

Huang G, Liu Z, Pleiss G, Maaten L, Weinberger K . Convolutional Networks with Dense Connectivity. IEEE Trans Pattern Anal Mach Intell. 2019; 44(12):8704-8716. DOI: 10.1109/TPAMI.2019.2918284. View

15.

Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O . Highly accurate protein structure prediction with AlphaFold. Nature. 2021; 596(7873):583-589. PMC: 8371605. DOI: 10.1038/s41586-021-03819-2. View

16.

Wood C, Heal J, Thomson A, Bartlett G, Ibarra A, Brady R . ISAMBARD: an open-source computational environment for biomolecular analysis, modelling and design. Bioinformatics. 2017; 33(19):3043-3050. PMC: 5870769. DOI: 10.1093/bioinformatics/btx352. View

17.

Huang X, Pearce R, Zhang Y . EvoEF2: accurate and fast energy function for computational protein design. Bioinformatics. 2019; 36(4):1135-1142. PMC: 7144094. DOI: 10.1093/bioinformatics/btz740. View

18.

Ludwiczak J, Jarmula A, Dunin-Horkawicz S . Combining Rosetta with molecular dynamics (MD): A benchmark of the MD-based ensemble protein design. J Struct Biol. 2018; 203(1):54-61. DOI: 10.1016/j.jsb.2018.02.004. View

19.

Yue K, Dill K . Inverse protein folding problem: designing polymer sequences. Proc Natl Acad Sci U S A. 1992; 89(9):4163-7. PMC: 525653. DOI: 10.1073/pnas.89.9.4163. View

20.

Knudsen M, Wiuf C . The CATH database. Hum Genomics. 2010; 4(3):207-12. PMC: 3525972. DOI: 10.1186/1479-7364-4-3-207. View