Computational Investigation of the Potential and Limitations of Machine Learning with Neural Network Circuits Based on Synaptic Transistors

Overview

Journal J Phys Chem Lett

Publisher American Chemical Society

Specialty Chemistry

Date 2024 Jun 28

PMID 38941557

Authors

Sergei Manzhos

Qun Gao Chen

Wen-Ya Lee

Yoon Heejoo

Manabu Ihara

Chu-Chen Chueh

Affiliations

Soon will be listed here.

Abstract

Synaptic transistors have been proposed to implement neuron activation functions of neural networks (NNs). While promising to enable compact, fast, inexpensive, and energy-efficient dedicated NN circuits, they also have limitations compared to digital NNs (realized as codes for digital processors), including shape choices of the activation function using particular types of transistor implementation, and instabilities due to noise and other factors present in analog circuits. We present a computational study of the effects of these factors on NN performance and find that, while accuracy competitive with traditional NNs can be realized for many applications, there is high sensitivity to the instability in the shape of the activation function, suggesting that, when highly accurate NNs are required, high-precision circuitry should be developed beyond what has been reported for synaptic transistors to date.

References

Manzhos S, Ihara M . Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension. J Chem Phys. 2024; 160(2). DOI: 10.1063/5.0187867. View

Manzhos S, Carrington Jr T . Using neural networks to represent potential surfaces as sums of products. J Chem Phys. 2006; 125(19):194105. DOI: 10.1063/1.2387950. View

Schutt K, Gastegger M, Tkatchenko A, Muller K, Maurer R . Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nat Commun. 2019; 10(1):5024. PMC: 6858523. DOI: 10.1038/s41467-019-12875-2. View

Wang Q, Zhao C, Sun Y, Xu R, Li C, Wang C . Synaptic transistor with multiple biological functions based on metal-organic frameworks combined with the LIF model of a spiking neural network to recognize temporal information. Microsyst Nanoeng. 2023; 9:96. PMC: 10362020. DOI: 10.1038/s41378-023-00566-4. View

Sung J, Chung S, Jang Y, Jang H, Kim J, Lee C . Unveiling the Role of Side Chain for Improving Nonvolatile Characteristics of Conjugated Polymers-Based Artificial Synapse. Adv Sci (Weinh). 2024; 11(16):e2400304. PMC: 11040376. DOI: 10.1002/advs.202400304. View

Manzhos S, Wang X, Dawes R, Carrington Jr T . A nested molecule-independent neural network approach for high-quality potential fits. J Phys Chem A. 2006; 110(16):5295-304. DOI: 10.1021/jp055253z. View

Yang Y, Wu Y, He W, Tien H, Yang W, Michinobu T . Tuning Ambipolarity of the Conjugated Polymer Channel Layers of Floating-Gate Free Transistors: From Volatile Memories to Artificial Synapses. Adv Sci (Weinh). 2022; 9(31):e2203025. PMC: 9631064. DOI: 10.1002/advs.202203025. View

Choi E, Jo J, Kim W, Min K . Searching for Mechanically Superior Solid-State Electrolytes in Li-Ion Batteries Data-Driven Approaches. ACS Appl Mater Interfaces. 2021; 13(36):42590-42597. DOI: 10.1021/acsami.1c07999. View

Wang G, Fearn T, Wang T, Choy K . Machine-Learning Approach for Predicting the Discharging Capacities of Doped Lithium Nickel-Cobalt-Manganese Cathode Materials in Li-Ion Batteries. ACS Cent Sci. 2021; 7(9):1551-1560. PMC: 8461773. DOI: 10.1021/acscentsci.1c00611. View

10.

Perdew , Burke , Ernzerhof . Generalized Gradient Approximation Made Simple. Phys Rev Lett. 1996; 77(18):3865-3868. DOI: 10.1103/PhysRevLett.77.3865. View

11.

Manzhos S, Ihara M . Neural Network with Optimal Neuron Activation Functions Based on Additive Gaussian Process Regression. J Phys Chem A. 2023; 127(37):7823-7835. DOI: 10.1021/acs.jpca.3c02949. View

12.

Yu T, Chen H, Liao M, Tien H, Chang T, Chueh C . Solution-Processable Anion-doped Conjugated Polymer for Nonvolatile Organic Transistor Memory with Synaptic Behaviors. ACS Appl Mater Interfaces. 2020; 12(30):33968-33978. DOI: 10.1021/acsami.0c06109. View

13.

Lin X, Rivenson Y, Yardimci N, Veli M, Luo Y, Jarrahi M . All-optical machine learning using diffractive deep neural networks. Science. 2018; 361(6406):1004-1008. DOI: 10.1126/science.aat8084. View

14.

Sahu H, Ma H . Unraveling Correlations between Molecular Properties and Device Parameters of Organic Solar Cells Using Machine Learning. J Phys Chem Lett. 2019; 10(22):7277-7284. DOI: 10.1021/acs.jpclett.9b02772. View

15.

Pour Fard M, Williamson I, Edwards M, Liu K, Pai S, Bartlett B . Experimental realization of arbitrary activation functions for optical neural networks. Opt Express. 2020; 28(8):12138-12148. DOI: 10.1364/OE.391473. View

16.

de Jong M, Chen W, Notestine R, Persson K, Ceder G, Jain A . A Statistical Learning Framework for Materials Science: Application to Elastic Moduli of k-nary Inorganic Polycrystalline Compounds. Sci Rep. 2016; 6:34256. PMC: 5046120. DOI: 10.1038/srep34256. View

17.

Unke O, Meuwly M . Toolkit for the Construction of Reproducing Kernel-Based Representations of Data: Application to Multidimensional Potential Energy Surfaces. J Chem Inf Model. 2017; 57(8):1923-1931. DOI: 10.1021/acs.jcim.7b00090. View

18.

Vandermause J, Xie Y, Lim J, Owen C, Kozinsky B . Active learning of reactive Bayesian force fields applied to heterogeneous catalysis dynamics of H/Pt. Nat Commun. 2022; 13(1):5183. PMC: 9440250. DOI: 10.1038/s41467-022-32294-0. View

19.

Nguyen H, Le H . Modified feed-forward neural network structures and combined-function-derivative approximations incorporating exchange symmetry for potential energy surface fitting. J Phys Chem A. 2012; 116(18):4629-38. DOI: 10.1021/jp3020386. View

20.

Kim S, Jeong Y, Bidenko P, Lim H, Jeon Y, Kim H . 3D Stackable Synaptic Transistor for 3D Integrated Artificial Neural Networks. ACS Appl Mater Interfaces. 2020; 12(6):7372-7380. DOI: 10.1021/acsami.9b22008. View