Unveiling the role of plasticity rules in reservoir computing
Morales, Guillermo B.; Mirasso, Claudio R.; Soriano, Miguel C.
Neurocomputing 461, 705-715 (2021)
Neurocomputing 461, 705-715 (2021)
Reservoir Computing (RC) is an appealing approach in Machine Learning that combines the high computational capabilities of Recurrent Neural Networks with a fast and easy training method. Likewise, successful implementation of neuro-inspired plasticity rules into RC artificial networks has boosted the performance of the original models. In this manuscript, we analyze the role that plasticity rules play on the changes that lead to a better performance of RC. To this end, we implement synaptic and non-synaptic plasticity rules in a paradigmatic example of RC model: the Echo State Network. Testing on nonlinear time series prediction tasks, we show evidence that improved performance in all plastic models are linked to a decrease of the pair-wise correlations in the reservoir, as well as a significant increase of individual neurons ability to separate similar inputs in their activity space. From the perspective of the reservoir dynamics, optimal performance is found to occur close to the edge of the instability. Here we provide new insights on this observed improvement through the study of different stages on the plastic learning. Our results also show that it is possible to combine different forms of plasticity (namely synaptic and non-synaptic rules) to further improve the performance on prediction tasks, obtaining better results than those achieved with single-plasticity models.
Related research projects
ADOPD
Adaptive Optical Dendrites
P.I.: Ingo Fischer, Claudio Mirasso
The increased demand for computation with low energy consumption requires entirely novel hardware concepts. In ADOPD we develop ultra-fast computational units based on optical-fiber technologies exploiting information processing principles used by neurons …
ADOPD (UIB)
Adaptive Optical Dendrites (IFISC-UIB)
P.I.: Claudio Mirasso
The increased demand for computation with low energy consumption requires entirely novel hardware concepts. In ADOPD we develop ultra-fast computational units based on optical-fiber technologies exploiting information processing principles used by neurons …
IDEA
Improving data DEcoding in optical communication networks All-optically using neuro-inspired photonic systems
P.I.: Miguel C. Soriano, Ingo Fischer, Claudio Mirasso
Novel technologies related to optical communications, sensing, the Internet of Things (IoT) and artificial intelligence have been generating unique opportunities and potential to enhance our quality-of-life, and to provide new services for …
IDEA (UIB)
Improving data DEcoding in optical communication networks All-optically using neuro-inspired photonic systems
P.I.: Miguel C. Soriano, Claudio Mirasso
Novel technologies related to optical communications, sensing, the Internet of Things (IoT) and artificial intelligence have been generating unique opportunities and potential to enhance our quality-of-life, and to provide new services for …