Statistical Mechanics of Neural Networks

Haiping Huang Dr. Haiping Huang received his Ph.D. degree in theoretical physics from the Institute of Theoretical Physics, the Chinese Academy of Sciences. He works as an associate professor at the School of Physics, Sun Yat-sen University, China. His research interests include the origin of the computational hardness of the binary perceptron model, the theory of dimension reduction in deep neural networks, and inherent symmetry breaking in unsupervised learning. In 2021, he was awarded Excellent Young Scientists Fund by National Natural Science Foundation of China.

Introduction.- Spin glass models and cavity method.- Variational mean-field theory and belief propagation.- Monte Carlo simulation methods.- High-temperature expansion.- Nishimori line.- Random energy model.- Statistical mechanical theory of Hopfield model.-  Replica symmetry and replica symmetry breaking.- Statistical mechanics of restricted Boltzmann machine.- Simplest model of unsupervised learning with binary synapses.-  Inherent-symmetry breaking in unsupervised learning.- Mean-field theory of Ising Perceptron.- Mean-field model of multi-layered Perceptron.- Mean-field theory of dimension reduction.- Chaos theory of random recurrent neural networks.- Statistical mechanics of random matrices.- Perspectives.