SQUIDS Seminar (ProbAI-sponsored): Adaptive friction and nonlinear damping for model training
| Dates: | 29 October 2025 |
|---|
| Times: | 13:00 - 14:00 |
|---|
| What is it: | Seminar |
|---|
| Organiser: | Department of Mathematics |
|---|
| Who is it for: | University staff, Current University students |
|---|
| Speaker: | Katerina Karoni |
|---|
|
Speaker: Katerina Karoni (Bristol)
Abstract: We discuss novel damping procedures for training large scale Bayesian data models, such as deep neural networks. Drawing inspiration from the concept of a thermostat (widely used for temperature regulation in molecular dynamics), we introduce kinetic energy controls on the individual parameter velocities of the model. This approach can be likened to component-wise Nosé-Hoover style thermostatting taken in the zero-temperature limit and it can be directly related to the introduction of cubic damping, a vibration suppression mechanism used in structural engineering applications. While a large momentum parameter helps to overcome barriers and progress training in low-curvature regions, it should be reduced in areas with steep gradients to avoid instability; our adaptive scheme allows this adjustment to be performed automatically, on a per-parameter basis. By using these schemes, we obtain enhanced efficiency, including significant speedups and test accuracy improvements in representative deep learning tasks.
Speaker
Katerina Karoni
Organisation: University of Bristol
Travel and Contact Information
Find event
Frank Adams Room 2
Alan Turing Building
Manchester
