On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Overview

Journal PLoS One

Specialties General Medicine
Science

Date 2021 Jan 14

PMID 33444316

Citations 4

Authors

Vladimir Kunc

Jiri Klema

Affiliations

Soon will be listed here.

Abstract

Gene expression profiling was made more cost-effective by the NIH LINCS program that profiles only ∼1, 000 selected landmark genes and uses them to reconstruct the whole profile. The D-GEX method employs neural networks to infer the entire profile. However, the original D-GEX can be significantly improved. We propose a novel transformative adaptive activation function that improves the gene expression inference even further and which generalizes several existing adaptive activation functions. Our improved neural network achieves an average mean absolute error of 0.1340, which is a significant improvement over our reimplementation of the original D-GEX, which achieves an average mean absolute error of 0.1637. The proposed transformative adaptive function enables a significantly more accurate reconstruction of the full gene expression profiles with only a small increase in the complexity of the model and its training procedure compared to other methods.

Citing Articles

Gene expression prediction based on neighbour connection neural network utilizing gene interaction graphs.

Li X, Zhang X, He W, Bu D, Zhang S PLoS One. 2023; 18(2):e0281286.

PMID: 36745614 PMC: 9901809. DOI: 10.1371/journal.pone.0281286.

Adaptive cognition implemented with a context-aware and flexible neuron for next-generation artificial intelligence.

Jadaun P, Cui C, Liu S, Incorvia J PNAS Nexus. 2023; 1(5):pgac206.

PMID: 36712357 PMC: 9802372. DOI: 10.1093/pnasnexus/pgac206.

On tower and checkerboard neural network architectures for gene expression inference.

Kunc V, Klema J BMC Genomics. 2020; 21(Suppl 5):454.

PMID: 33327945 PMC: 7739475. DOI: 10.1186/s12864-020-06821-6.

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks.

Jagtap A, Kawaguchi K, Karniadakis G Proc Math Phys Eng Sci. 2020; 476(2239):20200334.

PMID: 32831616 PMC: 7426042. DOI: 10.1098/rspa.2020.0334.

References

Lancashire L, Lemetre C, Ball G . An introduction to artificial neural networks in bioinformatics--application to complex microarray and mass spectrometry datasets in cancer studies. Brief Bioinform. 2009; 10(3):315-29. DOI: 10.1093/bib/bbp012. View

Guarnieri S, Piazza F, Uncini A . Multilayer feedforward networks with adaptive spline activation function. IEEE Trans Neural Netw. 2008; 10(3):672-83. DOI: 10.1109/72.761726. View

Ritchie M, Phipson B, Wu D, Hu Y, Law C, Shi W . limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res. 2015; 43(7):e47. PMC: 4402510. DOI: 10.1093/nar/gkv007. View

Yamada T, Yabuta T . Neural network controller using autotuning method for nonlinear functions. IEEE Trans Neural Netw. 1992; 3(4):595-601. DOI: 10.1109/72.143373. View

Goh S, Mandic D . Recurrent neural networks with trainable amplitude of activation functions. Neural Netw. 2003; 16(8):1095-100. DOI: 10.1016/S0893-6080(03)00139-4. View

Edsgard D, Johnsson P, Sandberg R . Identification of spatial expression trends in single-cell gene expression data. Nat Methods. 2018; 15(5):339-342. PMC: 6314435. DOI: 10.1038/nmeth.4634. View

Lappalainen T, Sammeth M, Friedlander M, t Hoen P, Monlong J, Rivas M . Transcriptome and genome sequencing uncovers functional variation in humans. Nature. 2013; 501(7468):506-11. PMC: 3918453. DOI: 10.1038/nature12531. View

Angermueller C, Parnamaa T, Parts L, Stegle O . Deep learning for computational biology. Mol Syst Biol. 2016; 12(7):878. PMC: 4965871. DOI: 10.15252/msb.20156651. View

Tiwari A, Srivastava R . A survey of computational intelligence techniques in protein function prediction. Int J Proteomics. 2015; 2014:845479. PMC: 4276698. DOI: 10.1155/2014/845479. View

10.

Wang H, Li C, Zhang J, Wang J, Ma Y, Lian Y . A new LSTM-based gene expression prediction model: L-GEPM. J Bioinform Comput Biol. 2019; 17(4):1950022. DOI: 10.1142/S0219720019500227. View

11.

Jagtap A, Kawaguchi K, Karniadakis G . Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proc Math Phys Eng Sci. 2020; 476(2239):20200334. PMC: 7426042. DOI: 10.1098/rspa.2020.0334. View

12.

Chen Y, Li Y, Narayan R, Subramanian A, Xie X . Gene expression inference with deep learning. Bioinformatics. 2016; 32(12):1832-9. PMC: 4908320. DOI: 10.1093/bioinformatics/btw074. View

13.

Medler D, Dawson M . Training redundant artificial neural networks: imposing biology on technology. Psychol Res. 1994; 57(1):54-62. DOI: 10.1007/BF00452996. View

14.

Min S, Lee B, Yoon S . Deep learning in bioinformatics. Brief Bioinform. 2016; 18(5):851-869. DOI: 10.1093/bib/bbw068. View

15.

Eetemadi A, Tagkopoulos I . Genetic Neural Networks: an artificial neural network architecture for capturing gene expression relationships. Bioinformatics. 2018; 35(13):2226-2234. DOI: 10.1093/bioinformatics/bty945. View

16.

Li W, Yin Y, Quan X, Zhang H . Gene Expression Value Prediction Based on XGBoost Algorithm. Front Genet. 2019; 10:1077. PMC: 6861218. DOI: 10.3389/fgene.2019.01077. View

17.

Subramanian A, Narayan R, Corsello S, Peck D, Natoli T, Lu X . A Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles. Cell. 2017; 171(6):1437-1452.e17. PMC: 5990023. DOI: 10.1016/j.cell.2017.10.049. View

18.

Nielsen J, Thorolfsdottir R, Fritsche L, Zhou W, Skov M, Graham S . Biobank-driven genomic discovery yields new insight into atrial fibrillation biology. Nat Genet. 2018; 50(9):1234-1239. PMC: 6530775. DOI: 10.1038/s41588-018-0171-3. View

19.

Wang X, Dizaji K, Huang H . Conditional generative adversarial network for gene expression inference. Bioinformatics. 2018; 34(17):i603-i611. PMC: 6129263. DOI: 10.1093/bioinformatics/bty563. View

20.

Hou L, Samaras D, Kurc T, Gao Y, Saltz J . ConvNets with Smooth Adaptive Activation Functions for Regression. Proc Mach Learn Res. 2019; 54:430-439. PMC: 6521876. View