Towards Understanding Deep Learning through the Lens of Statistical Physics

This is an online seminar. Registration is required.

【Date】Monday, September 29th, 14:00 - 15:00
【Speaker】Wei Huang, AIP Deep Learning Theory Team

TITLE:
Towards Understanding Deep Learning through the Lens of Statistical Physics

ABSTRACT:
Large foundation models such as GPT and stable diffusion models are transforming science and society. Despite their impressive capabilities, we still lack clear principles to explain why they generalize, when they fail, and how emergent abilities arise. To address these questions, I approach deep learning from the perspective of statistical physics, viewing neural networks as complex systems with many interacting components. This allows us to apply tools such as mean-field theory and phase transitions to reveal hidden laws of learning. In this talk, I will first introduce kernel methods as a mean-field description of infinite-width networks, showing how neural tangent kernels help explain the role of orthogonal initialization and how graph neural tangent kernels clarify the trainability of graph neural networks. I will then present a framework of feature learning theory, where signal–noise models serve as a foundational model for deep learning, analogous to the Ising model in statistical physics. This framework explains phenomena such as benign overfitting in Transformers and provides insights into feature learning in diffusion models. Together, these studies show how statistical-physics-inspired approaches can uncover hidden principles of deep learning and foundation models.