One-Dimensional Embeddings of Neural Network Training Dynamics
Artificial neural networks define high-dimensional nonlinear dynamical systems whose optimization processes remain only partially understood. Understanding these dynamics is particularly challenging because training trajectories evolve in parameter spaces with tens of thousands or even millions of dimensions, making their direct visualization and characterization difficult. In this talk, I will present a scalar embedding methodology that maps neural network training trajectories onto a one dimensional representation while preserving relevant dynamical information. Using a multilayer perceptron trained on the MNIST classification task, I will show how the embedding captures different learning regimes, including the emergence of sensitivity to initial conditions, and how it enables the definition of dynamical observables such as trajectory decorrelation times. I will also discuss how the embedding provides insight into the organization of asymptotic training states, revealing nontrivial statistical regularities in the distribution of minima explored during optimization.
This Annual PhD student seminar will be broadcasted in the following zoom link: https://us06web.zoom.us/j/89027654460?pwd=Wg9TYMPqqP2ipfj2JVvEagmzaTw29c.1
Detalles de contacto:
Miguel C. Soriano Contact form