A Unified Framework for Reservoir Computing and Extreme Learning Machines Based on a Single Time-delayed Neuron

Overview

Journal Sci Rep

Specialty Science

Date 2015 Oct 9

PMID 26446303

Citations 19

Authors

S Ortin

M C Soriano

L Pesquera

D Brunner

D San-Martin

I Fischer

C R Mirasso

J M Gutierrez

Affiliations

Soon will be listed here.

Abstract

In this paper we present a unified framework for extreme learning machines and reservoir computing (echo state networks), which can be physically implemented using a single nonlinear neuron subject to delayed feedback. The reservoir is built within the delay-line, employing a number of "virtual" neurons. These virtual neurons receive random projections from the input layer containing the information to be processed. One key advantage of this approach is that it can be implemented efficiently in hardware. We show that the reservoir computing implementation, in this case optoelectronic, is also capable to realize extreme learning machines, demonstrating the unified framework for both schemes in software as well as in hardware.

Citing Articles

Efficient optimisation of physical reservoir computers using only a delayed input.

Picco E, Jaurigue L, Ludge K, Massar S Commun Eng. 2025; 4(1):3.

PMID: 39827312 PMC: 11742992. DOI: 10.1038/s44172-025-00340-6.

Photonic neuromorphic technologies in optical communications.

Argyris A Nanophotonics. 2024; 11(5):897-916.

PMID: 39634468 PMC: 11501306. DOI: 10.1515/nanoph-2021-0578.

Parallel and deep reservoir computing using semiconductor lasers with optical feedback.

Hasegawa H, Kanno K, Uchida A Nanophotonics. 2024; 12(5):869-881.

PMID: 39634361 PMC: 11501584. DOI: 10.1515/nanoph-2022-0440.

Complex dynamics in nonlinear small time-delayed optoelectronic oscillator and application in fast reservoir computing and pulse generation.

Tang D, Liang E, Lu Q, Zhao H, Li Z Sci Rep. 2024; 14(1):18672.

PMID: 39134624 PMC: 11319353. DOI: 10.1038/s41598-024-69585-z.

Pattern recognition in reciprocal space with a magnon-scattering reservoir.

Korber L, Heins C, Hula T, Kim J, Thlang S, Schultheiss H Nat Commun. 2023; 14(1):3954.

PMID: 37402733 PMC: 10319722. DOI: 10.1038/s41467-023-39452-y.

References

Brunner D, Soriano M, Mirasso C, Fischer I . Parallel photonic information processing at gigabyte per second data rates using transient states. Nat Commun. 2013; 4:1364. PMC: 3562454. DOI: 10.1038/ncomms2368. View

Soriano M, Ortin S, Brunner D, Larger L, Mirasso C, Fischer I . Optoelectronic reservoir computing: tackling noise-induced performance degradation. Opt Express. 2013; 21(1):12-20. DOI: 10.1364/OE.21.000012. View

Zhang B, Miller D, Wang Y . Nonlinear system modeling with random matrices: echo state networks revisited. IEEE Trans Neural Netw Learn Syst. 2014; 23(1):175-82. PMC: 4107715. DOI: 10.1109/TNNLS.2011.2178562. View

Huang G, Huang G, Song S, You K . Trends in extreme learning machines: a review. Neural Netw. 2014; 61:32-48. DOI: 10.1016/j.neunet.2014.10.001. View

Appeltant L, Soriano M, Van der Sande G, Danckaert J, Massar S, Dambre J . Information processing using a single dynamical node as complex system. Nat Commun. 2011; 2:468. PMC: 3195233. DOI: 10.1038/ncomms1476. View

Duport F, Schneider B, Smerieri A, Haelterman M, Massar S . All-optical reservoir computing. Opt Express. 2012; 20(20):22783-95. DOI: 10.1364/OE.20.022783. View

Jaeger H, Haas H . Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004; 304(5667):78-80. DOI: 10.1126/science.1091277. View

Rodan A, Tino P . Simple deterministically constructed cycle reservoirs with regular jumps. Neural Comput. 2012; 24(7):1822-52. DOI: 10.1162/NECO_a_00297. View

Rodan A, Tino P . Minimum complexity echo state network. IEEE Trans Neural Netw. 2010; 22(1):131-44. DOI: 10.1109/TNN.2010.2089641. View

10.

Dambre J, Verstraeten D, Schrauwen B, Massar S . Information processing capacity of dynamical systems. Sci Rep. 2012; 2:514. PMC: 3400147. DOI: 10.1038/srep00514. View

11.

Larger L, Soriano M, Brunner D, Appeltant L, Gutierrez J, Pesquera L . Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. Opt Express. 2012; 20(3):3241-9. DOI: 10.1364/OE.20.003241. View

12.

Butcher J, Verstraeten D, Schrauwen B, Day C, Haycock P . Reservoir computing and extreme learning machines for non-linear time-series data analysis. Neural Netw. 2013; 38:76-89. DOI: 10.1016/j.neunet.2012.11.011. View

13.

Dagliyan O, Uney-Yuksektepe F, Kavakli I, Turkay M . Optimization based tumor classification from microarray gene expression data. PLoS One. 2011; 6(2):e14579. PMC: 3033885. DOI: 10.1371/journal.pone.0014579. View

14.

Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, Haelterman M . Optoelectronic reservoir computing. Sci Rep. 2012; 2:287. PMC: 3286854. DOI: 10.1038/srep00287. View