Skip to content
Maths for Deep LearningResearch directory

Research themes

From representation
to reliable prediction

Explore the mathematical questions behind deep learning, from the functions a model can express to its behaviour on unfamiliar data.

Three questions that belong together

Approximation, optimisation and generalisation describe different parts of learning. A function may be representable yet difficult to learn; a training procedure may fit the observations yet perform poorly on a new distribution.

Representation

Approximation and structure

Depth and width influence the class of functions available to a network. Useful questions concern approximation error at a fixed parameter budget, regularity of the target, and whether known symmetries can reduce unnecessary flexibility.

Algorithms

Optimisation and dynamics

The loss surface, parameterisation and update rule jointly shape training. Track sensitivity to initialisation, learning rate and batch size. Similar final training losses can conceal different trajectories and different predictive behaviour.

Evidence

Generalisation and stability

Separate error on a finite test sample from performance on the target population. Study how sample size, model complexity and regularisation interact, and state assumptions about independence and distribution shift explicitly.

Learning with physical structure

In scientific computing, observations often sit alongside equations, boundary conditions and conservation principles. These can inform a model architecture, a training objective or an evaluation procedure. Each choice creates a different mathematical problem.

A small equation residual at selected points does not, by itself, establish a small solution error everywhere. The relationship depends on the equation, sampling, boundary treatment and numerical stability. Compare a learned solution with a trusted reference and test beyond the training configurations.

Inverse problems

Recovering what cannot be observed directly

Inverse problems infer hidden quantities from indirect, often noisy measurements. A learned reconstruction method can encode useful prior information, but that information also influences which solutions it favours.

Useful analysis asks whether the measurements determine a unique answer, how noise changes the reconstruction, and what happens when the test data differ from the training examples. Uncertainty should be connected to a stated source of variation rather than presented as a single unexplained score.

Design an experiment that answers one question

State the claim and assumptions

Specify the data distribution, target function, architecture and error measure. Make the proposed mathematical relationship precise enough to be challenged.

Choose a credible comparison

Include a simple baseline and, where appropriate, a conventional numerical method. Keep access to data and the computational budget comparable, or report the differences.

Test sensitivity and failure

Repeat runs with different random seeds. Vary noise, resolution or sample size, and record unsuccessful settings as carefully as successful ones.

Read the accompanying notes