×

Decoupled echo state networks with lateral inhibition. (English) Zbl 1132.68610

Summary: Building on some prior work, in this paper we describe a novel structure termed the Decoupled Echo State Network (DESN) involving the use of lateral inhibition. Two low-complexity implementation schemes, namely, the DESN with reservoir prediction (DESN + RP) and DESN with maximum available information (DESN + MaxInfo), are developed: (1) In the multiple superimposed oscillator problem, DESN + MaxInfo exhibits three important attributes: lower generalization mean-square error, better robustness with respect to the random generation of reservoir weight matrix and feedback connections, and robustness to variations in the sparseness of reservoir weight matrix, compared to DESN + RP. (2) For a noiseless nonlinear prediction task, DESN + RP outperforms the DESN + MaxInfo and single reservoir-based ESN approach in terms of lower prediction MSE and better robustness to a change in the number of inputs and sparsity of the reservoir weight matrix.Finally, in a real-life prediction task using noisy sea clutter data, both schemes exhibit higher prediction accuracy and successful design ratio than a conventional ESN with a single reservoir.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

Evolino
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Arbib, M. A., The metaphorical brain (1989), Wiley: Wiley New York · Zbl 1079.92500
[2] Bush, K., & Tsendjav, B. (2005). Improving the richness of echo state features using next ascent local search. In Proc. artificial neural networks in engineering conf.; Bush, K., & Tsendjav, B. (2005). Improving the richness of echo state features using next ascent local search. In Proc. artificial neural networks in engineering conf.
[3] Diamantaras, K. I.; Kung, S. K., Principal component neural network: Theory and application (1996), Wiley: Wiley New York · Zbl 0911.68157
[4] Fischler, M. A.; Firschein, O., Intelligence: The eye, the brain and the computer (1987), Addison-Wesley: Addison-Wesley Reading, MA
[5] Haykin, S., Neural networks: A comprehensive foundation (2004), Prentice-Hall PTR: Prentice-Hall PTR Upper Saddle River, NJ · Zbl 0828.68103
[6] Ishii, K., van der Zant, T., Becanovic, V., & Plöger, P. (2004). Identification of motion with echo state network. In Proc. OCEANS’04; Ishii, K., van der Zant, T., Becanovic, V., & Plöger, P. (2004). Identification of motion with echo state network. In Proc. OCEANS’04
[7] Jaeger, H. (2001). The echo state approach to analysing and training recurrent neural networks. GMD report 148; Jaeger, H. (2001). The echo state approach to analysing and training recurrent neural networks. GMD report 148
[8] Jaeger, H., Adaptive nonlinear system identification with echo state network, (Derickson, D., Advances in neural information processing systems, Vol. 15 (2003), MIT Press: MIT Press Cambridge, MA), 593-600
[9] Jaeger, H., Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science, 304, 18-80 (2004)
[10] Jaeger, H. (2004b). Seminar slides. (Online) Available http://www.faculty.iu-bremen.de/hjaeger/courses/Seminar Spring04/ESNStandardSlides.pdf; Jaeger, H. (2004b). Seminar slides. (Online) Available http://www.faculty.iu-bremen.de/hjaeger/courses/Seminar Spring04/ESNStandardSlides.pdf
[11] Maass, W.; Natschläger, T.; Markram, H., Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Computation, 14, 11, 2531-2560 (2002) · Zbl 1057.68618
[12] Plöger, P. G., Arghir, A., Günther, T., & Hosseiny, R. (2003). Echo state networks for mobile robot modeling and control. In Proc. RoboCup; Plöger, P. G., Arghir, A., Günther, T., & Hosseiny, R. (2003). Echo state networks for mobile robot modeling and control. In Proc. RoboCup
[13] Prokhorov, D. (2005). Echo state networks: Appeal and challenges. In Proc. of international joint conference on neural networks; Prokhorov, D. (2005). Echo state networks: Appeal and challenges. In Proc. of international joint conference on neural networks
[14] Schmidhuber, J., Gagliolo, M., Wierstra, D., & Gomez, F. (2005). Evolino for recurrent support vector machines. Technical report no. IDSIA-19-15 (version 2.0); Schmidhuber, J., Gagliolo, M., Wierstra, D., & Gomez, F. (2005). Evolino for recurrent support vector machines. Technical report no. IDSIA-19-15 (version 2.0) · Zbl 1127.68085
[15] Werbos, P. J., Backpropagation through time: What it does and how to do it, Proceedings of the IEEE, 78, 10, 1550-1560 (1990)
[16] Wiestra, D., Gomez, F., & Schmidhuber, J. (2005). Modeling systems with internal state using Evolino. In GECCO’05; Wiestra, D., Gomez, F., & Schmidhuber, J. (2005). Modeling systems with internal state using Evolino. In GECCO’05
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.