SQUIDS Seminar: Optimizing the Dynamics of the Frequency Bias in Fourier Features Neural Networks
| Dates: | 29 April 2026 |
|---|
| Times: | 13:00 - 14:00 |
|---|
| What is it: | Seminar |
|---|
| Organiser: | Department of Mathematics |
|---|
| Who is it for: | University staff, External researchers, Current University students |
|---|
| Speaker: | Matias Courdurier |
|---|
|
|
Speaker: Professor Matias Courdurier (Pontifical Catholic University of Chile)
Abstract:
Implicit Neural Representations often learn signals capturing low-frequency structure before high-frequency details, a phenomenon known as Frequency Bias. Fourier Feature networks can reduce this effect by encoding inputs with sinusoidal components sampled from a prescribed distribution.
In this talk we will present how, in the Neural Tangent Kernel setting, it is possible to derive a PDE that describes the evolution, during training, of different frequencies and use the exact form of the PDE to select initialization distributions to tune or suppress this frequency bias. We formulate an explicit optimization procedure to pick an optimal first-layer parameter distribution, including a Gaussian-restricted version, yielding faster early-stage convergence. Through experiments we also validate the theorical prediction and, by testing across tasks and deeper FF architectures, we show robustness of the proposed approach beyond the simplified setting in which the PDE is obtained.
This is a joint work with Juan Jose Molina, Mircea Petrache and Francisco Sahli Costabal.
Speaker
Matias Courdurier
Organisation: Pontifical Catholic University of Chile
Travel and Contact Information
Find event
Frank Adams Room 2
Alan Turing Building
Manchester
