On Practical Issues for Stochastic STDP Hardware With 1-bit Synaptic Weights

Overview

Journal Front Neurosci

Date 2018 Oct 31

PMID 30374283

Citations 12

Authors

Amirreza Yousefzadeh

Evangelos Stromatias

Miguel Soto

Teresa Serrano-Gotarredona

Bernabe Linares-Barranco

Affiliations

Soon will be listed here.

Abstract

In computational neuroscience, synaptic plasticity learning rules are typically studied using the full 64-bit floating point precision computers provide. However, for dedicated hardware implementations, the precision used not only penalizes directly the required memory resources, but also the computing, communication, and energy resources. When it comes to hardware engineering, a key question is always to find the minimum number of necessary bits to keep the neurocomputational system working satisfactorily. Here we present some techniques and results obtained when limiting synaptic weights to 1-bit precision, applied to a Spike-Timing-Dependent-Plasticity (STDP) learning rule in Spiking Neural Networks (SNN). We first illustrate the 1-bit synapses STDP operation by replicating a classical biological experiment on visual orientation tuning, using a simple four neuron setup. After this, we apply 1-bit STDP learning to the hidden feature extraction layer of a 2-layer system, where for the second (and output) layer we use already reported SNN classifiers. The systems are tested on two spiking datasets: a Dynamic Vision Sensor (DVS) recorded poker card symbols dataset and a Poisson-distributed spike representation MNIST dataset version. Tests are performed using the in-house MegaSim event-driven behavioral simulator and by implementing the systems on FPGA (Field Programmable Gate Array) hardware.

Citing Articles

Overview of Spiking Neural Network Learning Approaches and Their Computational Complexities.

Pietrzak P, Szczesny S, Huderek D, Przyborowski L Sensors (Basel). 2023; 23(6).

PMID: 36991750 PMC: 10053242. DOI: 10.3390/s23063037.

A CMOS-memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity.

Ahmadi-Farsani J, Ricci S, Hashemkhani S, Ielmini D, Linares-Barranco B, Serrano-Gotarredona T Philos Trans A Math Phys Eng Sci. 2022; 380(2228):20210018.

PMID: 35658675 PMC: 9168445. DOI: 10.1098/rsta.2021.0018.

Application of Neuromorphic Olfactory Approach for High-Accuracy Classification of Malts.

Vanarse A, Osseiran A, Rassau A, van der Made P Sensors (Basel). 2022; 22(2).

PMID: 35062402 PMC: 8778084. DOI: 10.3390/s22020440.

Stochastic binary synapses having sigmoidal cumulative distribution functions for unsupervised learning with spike timing-dependent plasticity.

Nishi Y, Nomura K, Marukame T, Mizushima K Sci Rep. 2021; 11(1):18282.

PMID: 34521895 PMC: 8440757. DOI: 10.1038/s41598-021-97583-y.

Using a Low-Power Spiking Continuous Time Neuron (SCTN) for Sound Signal Processing.

Bensimon M, Greenberg S, Haiut M Sensors (Basel). 2021; 21(4).

PMID: 33557214 PMC: 7913968. DOI: 10.3390/s21041065.

References

Mostafa H . Supervised Learning Based on Temporal Coding in Spiking Neural Networks. IEEE Trans Neural Netw Learn Syst. 2017; 29(7):3227-3235. DOI: 10.1109/TNNLS.2017.2726060. View

Feldman D . Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron. 2000; 27(1):45-56. DOI: 10.1016/s0896-6273(00)00008-8. View

Stromatias E, Soto M, Serrano-Gotarredona T, Linares-Barranco B . An Event-Driven Classifier for Spiking Neural Networks Fed with Synthetic or Dynamic Vision Sensor Data. Front Neurosci. 2017; 11:350. PMC: 5487436. DOI: 10.3389/fnins.2017.00350. View

Mu Y, Poo M . Spike timing-dependent LTP/LTD mediates visual experience-dependent plasticity in a developing retinotectal system. Neuron. 2006; 50(1):115-25. DOI: 10.1016/j.neuron.2006.03.009. View

Orchard G, Meyer C, Etienne-Cummings R, Posch C, Thakor N, Benosman R . HFirst: A Temporal Approach to Object Recognition. IEEE Trans Pattern Anal Mach Intell. 2015; 37(10):2028-40. DOI: 10.1109/TPAMI.2015.2392947. View

Kheradpisheh S, Ganjtabesh M, Thorpe S, Masquelier T . STDP-based spiking deep convolutional neural networks for object recognition. Neural Netw. 2018; 99:56-67. DOI: 10.1016/j.neunet.2017.12.005. View

Masquelier T, Guyonneau R, Thorpe S . Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS One. 2008; 3(1):e1377. PMC: 2147052. DOI: 10.1371/journal.pone.0001377. View

Cassenaer S, Laurent G . Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature. 2007; 448(7154):709-13. DOI: 10.1038/nature05973. View

Perez-Carrasco J, Zhao B, Serrano C, Acha B, Serrano-Gotarredona T, Chen S . Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing--application to feedforward ConvNets. IEEE Trans Pattern Anal Mach Intell. 2013; 35(11):2706-19. DOI: 10.1109/TPAMI.2013.71. View

10.

Neftci E, Das S, Pedroni B, Kreutz-Delgado K, Cauwenberghs G . Event-driven contrastive divergence: neural sampling foundations. Front Neurosci. 2015; 9:104. PMC: 4379871. DOI: 10.3389/fnins.2015.00104. View

11.

Neftci E, Augustine C, Paul S, Detorakis G . Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines. Front Neurosci. 2017; 11:324. PMC: 5478701. DOI: 10.3389/fnins.2017.00324. View

12.

Diehl P, Cook M . Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci. 2016; 9:99. PMC: 4522567. DOI: 10.3389/fncom.2015.00099. View

13.

Jeyabalaratnam J, Bharmauria V, Bachatene L, Cattan S, Angers A, Molotchnikoff S . Adaptation shifts preferred orientation of tuning curve in the mouse visual cortex. PLoS One. 2013; 8(5):e64294. PMC: 3662720. DOI: 10.1371/journal.pone.0064294. View

14.

Zamarreno-Ramos C, Linares-Barranco A, Serrano-Gotarredona T, Linares-Barranco B . Multicasting mesh AER: a scalable assembly approach for reconfigurable neuromorphic structured AER systems. Application to ConvNets. IEEE Trans Biomed Circuits Syst. 2013; 7(1):82-102. DOI: 10.1109/TBCAS.2012.2195725. View

15.

Mozafari M, Kheradpisheh S, Masquelier T, Nowzari-Dalini A, Ganjtabesh M . First-Spike-Based Visual Categorization Using Reward-Modulated STDP. IEEE Trans Neural Netw Learn Syst. 2018; 29(12):6178-6190. DOI: 10.1109/TNNLS.2018.2826721. View

16.

Masquelier T, Thorpe S . Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput Biol. 2007; 3(2):e31. PMC: 1797822. DOI: 10.1371/journal.pcbi.0030031. View

17.

Moore 4th B, Freeman R . Development of orientation tuning in simple cells of primary visual cortex. J Neurophysiol. 2012; 107(9):2506-16. PMC: 3362253. DOI: 10.1152/jn.00719.2011. View

18.

Stromatias E, Neil D, Pfeiffer M, Galluppi F, Furber S, Liu S . Robustness of spiking Deep Belief Networks to noise and reduced bit precision of neuro-inspired hardware platforms. Front Neurosci. 2015; 9:222. PMC: 4496577. DOI: 10.3389/fnins.2015.00222. View

19.

Galluppi F, Lagorce X, Stromatias E, Pfeiffer M, Plana L, Furber S . A framework for plasticity implementation on the SpiNNaker neural architecture. Front Neurosci. 2015; 8:429. PMC: 4299433. DOI: 10.3389/fnins.2014.00429. View

20.

Zhang L, Tao H, Holt C, Harris W, Poo M . A critical window for cooperation and competition among developing retinotectal synapses. Nature. 1998; 395(6697):37-44. DOI: 10.1038/25665. View