Optimization and Sampling via Hamiltonian Mechanics
Time: December 15 (Tue.) 15:00-16:00
Venue: M212, Gongguan Campus, NTNU
Asst. Prof. Jun-Kun Wang
Department of Electrical and Computer Engineering
and the Halicioğlu Data Science Institute at UC San Diego
The laws of classical mechanics, which describe the dynamics of moving bodies in physical space, have been a subject of great practical interest and inspired centuries of deep mathematical study. In particular, the perspective that describes the dynamics of a particle as a Hamiltonian flow in phase space has found useful applications in designing algorithms such as Hamiltonian Monte Carlo (HMC) in sampling. In this talk, I will describe a scheme of time-varying integration time for HMC so that it can have a provable acceleration, compared to using a fixed constant integration time, for sampling from a class of distributions. I will then switch to optimization and introduce an optimization algorithm called Frictionless Hamiltonian Descent, which can be viewed as a direct counterpart of HMC in sampling. I will also cover Frictionless Coordinate Hamiltonian Descent and its parallelizable variant, which turns out to encapsulate the classical Gauss-Seidel method, Successive Over-relaxation, Jacobi method, and more, for solving a linear system of equations.
The talk will be based on the following two papers:
- Frictionless Hamiltonian Descent and Coordinate Hamiltonian Descent for Strongly Convex Quadratic Problems Journal of Optimization Theory and Applications 2026.
- Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time ICLR 2023. Joint work with Andre Wibisono.
More information: https://jimwang123.github.io/