Prediction of RNA-protein Interactions Using a Nucleotide Language Model

Overview

Journal Bioinform Adv

Publisher Oxford University Press

Specialty Biology

Date 2023 Jan 26

PMID 36699410

Authors

Keisuke Yamada

Michiaki Hamada

Affiliations

Soon will be listed here.

Abstract

Motivation: The accumulation of sequencing data has enabled researchers to predict the interactions between RNA sequences and RNA-binding proteins (RBPs) using novel machine learning techniques. However, existing models are often difficult to interpret and require additional information to sequences. Bidirectional encoder representations from transformer (BERT) is a language-based deep learning model that is highly interpretable. Therefore, a model based on BERT architecture can potentially overcome such limitations.

Results: Here, we propose BERT-RBP as a model to predict RNA-RBP interactions by adapting the BERT architecture pretrained on a human reference genome. Our model outperformed state-of-the-art prediction models using the eCLIP-seq data of 154 RBPs. The detailed analysis further revealed that BERT-RBP could recognize both the transcript region type and RNA secondary structure only based on sequence information. Overall, the results provide insights into the fine-tuning mechanism of BERT in biological contexts and provide evidence of the applicability of the model to other RNA-related problems.

Availability And Implementation: Python source codes are freely available at https://github.com/kkyamada/bert-rbp. The datasets underlying this article were derived from sources in the public domain: [RBPsuite (http://www.csbio.sjtu.edu.cn/bioinf/RBPsuite/), Ensembl Biomart (http://asia.ensembl.org/biomart/martview/)].

Supplementary Information: Supplementary data are available at online.

Citing Articles

RBPsuite 2.0: an updated RNA-protein binding site prediction suite with high coverage on species and proteins based on deep learning.

Pan X, Fang Y, Liu X, Guo X, Shen H BMC Biol. 2025; 23(1):74.

PMID: 40069726 PMC: 11899677. DOI: 10.1186/s12915-025-02182-2.

RNA sequence analysis landscape: A comprehensive review of task types, databases, datasets, word embedding methods, and language models.

Asim M, Ibrahim M, Asif T, Dengel A Heliyon. 2025; 11(2):e41488.

PMID: 39897847 PMC: 11783440. DOI: 10.1016/j.heliyon.2024.e41488.

RNAelem: an algorithm for discovering sequence-structure motifs in RNA bound by RNA-binding proteins.

Miyake H, Kawaguchi R, Kiryu H Bioinform Adv. 2024; 4(1):vbae144.

PMID: 39399375 PMC: 11471262. DOI: 10.1093/bioadv/vbae144.

scEGG: an exogenous gene-guided clustering method for single-cell transcriptomic data.

Hu D, Guan R, Liang K, Yu H, Quan H, Zhao Y Brief Bioinform. 2024; 25(6).

PMID: 39344711 PMC: 11440090. DOI: 10.1093/bib/bbae483.

Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.

AlSaad R, Abd-Alrazaq A, Boughorbel S, Ahmed A, Renault M, Damseh R J Med Internet Res. 2024; 26:e59505.

PMID: 39321458 PMC: 11464944. DOI: 10.2196/59505.

References

Strazar M, Zitnik M, Zupan B, Ule J, Curk T . Orthogonal matrix factorization enables integrative analysis of multiple RNA binding proteins. Bioinformatics. 2016; 32(10):1527-35. PMC: 4894278. DOI: 10.1093/bioinformatics/btw003. View

Maticzka D, Lange S, Costa F, Backofen R . GraphProt: modeling binding preferences of RNA-binding proteins. Genome Biol. 2014; 15(1):R17. PMC: 4053806. DOI: 10.1186/gb-2014-15-1-r17. View

Chung T, Kim D . Prediction of binding property of RNA-binding proteins using multi-sized filters and multi-modal deep convolutional neural network. PLoS One. 2019; 14(4):e0216257. PMC: 6485761. DOI: 10.1371/journal.pone.0216257. View

Rodriguez J, Maietta P, Ezkurdia I, Pietrelli A, Wesselink J, Lopez G . APPRIS: annotation of principal and alternative splice isoforms. Nucleic Acids Res. 2012; 41(Database issue):D110-7. PMC: 3531113. DOI: 10.1093/nar/gks1058. View

Pan X, Rijnbeek P, Yan J, Shen H . Prediction of RNA-protein sequence and structure binding preferences using deep convolutional and recurrent neural networks. BMC Genomics. 2018; 19(1):511. PMC: 6029131. DOI: 10.1186/s12864-018-4889-1. View

Ben-Bassat I, Chor B, Orenstein Y . A deep neural network approach for learning intrinsic protein-RNA binding preferences. Bioinformatics. 2018; 34(17):i638-i646. DOI: 10.1093/bioinformatics/bty600. View

Dominguez D, Freese P, Alexis M, Su A, Hochman M, Palden T . Sequence, Structure, and Context Preferences of Human RNA Binding Proteins. Mol Cell. 2018; 70(5):854-867.e9. PMC: 6062212. DOI: 10.1016/j.molcel.2018.05.001. View

Pan X, Shen H . Predicting RNA-protein binding sites and motifs through combining local and global deep convolutional neural networks. Bioinformatics. 2018; 34(20):3427-3436. DOI: 10.1093/bioinformatics/bty364. View

Zhang H, Zhang L, Mathews D, Huang L . LinearPartition: linear-time approximation of RNA folding partition function and base-pairing probabilities. Bioinformatics. 2020; 36(Suppl_1):i258-i267. PMC: 7355276. DOI: 10.1093/bioinformatics/btaa460. View

10.

Zhang S, Wang Y, Zhang X, Wang J . Prediction of the RBP binding sites on lncRNAs using the high-order nucleotide encoding convolutional neural network. Anal Biochem. 2019; 583:113364. DOI: 10.1016/j.ab.2019.113364. View

11.

Van Nostrand E, Freese P, Pratt G, Wang X, Wei X, Xiao R . A large-scale binding and functional map of human RNA-binding proteins. Nature. 2020; 583(7818):711-719. PMC: 7410833. DOI: 10.1038/s41586-020-2077-3. View

12.

Steinegger M, Soding J . MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nat Biotechnol. 2017; 35(11):1026-1028. DOI: 10.1038/nbt.3988. View

13.

Van Nostrand E, Pratt G, Shishkin A, Gelboin-Burkhart C, Fang M, Sundararaman B . Robust transcriptome-wide discovery of RNA-binding protein binding sites with enhanced CLIP (eCLIP). Nat Methods. 2016; 13(6):508-14. PMC: 4887338. DOI: 10.1038/nmeth.3810. View

14.

Geuens T, Bouhy D, Timmerman V . The hnRNP family: insights into their role in health and disease. Hum Genet. 2016; 135(8):851-67. PMC: 4947485. DOI: 10.1007/s00439-016-1683-5. View

15.

Licatalosi D, Mele A, Fak J, Ule J, Kayikci M, Chi S . HITS-CLIP yields genome-wide insights into brain alternative RNA processing. Nature. 2008; 456(7221):464-9. PMC: 2597294. DOI: 10.1038/nature07488. View

16.

Iuchi H, Matsutani T, Yamada K, Iwano N, Sumi S, Hosoda S . Representation learning applications in biological sequence analysis. Comput Struct Biotechnol J. 2021; 19:3198-3208. PMC: 8190442. DOI: 10.1016/j.csbj.2021.05.039. View

17.

Feng H, Bao S, Rahman M, Weyn-Vanhentenryck S, Khan A, Wong J . Modeling RNA-Binding Protein Specificity In Vivo by Precisely Registering Protein-RNA Crosslink Sites. Mol Cell. 2019; 74(6):1189-1204.e6. PMC: 6676488. DOI: 10.1016/j.molcel.2019.02.002. View

18.

Alipanahi B, Delong A, Weirauch M, Frey B . Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nat Biotechnol. 2015; 33(8):831-8. DOI: 10.1038/nbt.3300. View

19.

Yu H, Wang J, Sheng Q, Liu Q, Shyr Y . beRBP: binding estimation for human RNA-binding proteins. Nucleic Acids Res. 2018; 47(5):e26. PMC: 6411931. DOI: 10.1093/nar/gky1294. View

20.

Rao R, Bhattacharya N, Thomas N, Duan Y, Chen X, Canny J . Evaluating Protein Transfer Learning with TAPE. Adv Neural Inf Process Syst. 2021; 32:9689-9701. PMC: 7774645. View