Computing with Dynamical Systems: from implementations towards novel concepts

This thesis explores the computational capabilities of dynamical systems in the context of machine learning, aiming to enhance their efficiency in supervised tasks. By leveraging the inherent information-processing abilities of dynamical systems, it focuses on modeling photonic hardware as unconventional computing substrates and developing methods to improve optimization and generalization in recurrent neural networks (RNNs).
The study first examines the interaction of timescales in optoelectronic and photonic delay systems within the reservoir computing framework. It shows that aligning input signals with reservoir recurrence improves performance in chaotic time series prediction, an aspect previously overlooked in computing. These numerical findings are further validated with experimental data.
Despite the flexibility of physical substrates, its optimization potential has been underexploited. Comparing it with fully trained neural networks, the thesis finds that fully trained systems, despite having fewer nodes, achieve higher performance. To narrow the gap, the author proposes a regional training approach, which balances performance and the number of trainable parameters. A novel optimization method is introduced by formalizing the process as a dynamical system, using oscillatory weight dynamics to optimize physical computing substrates.
The thesis also addresses the generalization ability of dynamical systems, showing how they can adapt to new configurations and generate unseen temporal patterns. By leveraging model symmetry and the conceptor framework, the study demonstrates how dynamical systems can learn from limited data and infer complex motion patterns.
In conclusion, this work bridges dynamical systems, machine learning, and unconventional computing, presenting new methods to expand the computational power of reservoir computing systems and tackling more complex tasks.