Skip to content
Maths for Deep LearningResearch directory

Publications & research notes

Read closely.
Test the assumptions.

Short, self-contained notes connecting mathematical ideas to the design and interpretation of computational experiments.

Short notes with explicit limits

This collection contains introductory notes published on this site. They are explanations of research questions and experimental practice, rather than reports of new results or a bibliography of peer-reviewed papers. Each note is available in full below.

Separating sources of error

A trained model can miss its target for several reasons. The chosen function class may be too restrictive, the optimisation algorithm may not find a good solution within that class, or the available samples may poorly represent the population of interest.

For a fixed loss and population distribution, let the population risk be the expected loss on a fresh observation. Approximation error compares the best risk achievable in a chosen model class with an ideal reference. Statistical error concerns learning that choice from a finite sample. Optimisation error concerns how far the computed parameters remain from the chosen empirical objective’s optimum. The exact decomposition depends on the learning setup.

A practical diagnostic

Hold the data split fixed and increase the training budget. If the training loss falls but the test loss does not, more optimisation alone is unlikely to resolve the observed test error. Next vary the sample size and model class separately; changing all three at once makes the source of improvement difficult to identify.

Limit: these experiments provide evidence about a particular setup. They do not prove that the same explanation holds across architectures or data distributions.

Connect this note to the research themes →

Testing what physical structure contributes

Physical knowledge can enter a learning system through its inputs, architecture, loss or constraints. For example, a model may be designed to respect a transformation symmetry, or trained with a penalty for violating a governing equation.

These mechanisms should be evaluated separately. Compare a model with and without the structural component while keeping the remaining setup as close as possible. Report any change in parameter count, training time or access to information. Otherwise an apparent benefit of structure may reflect additional computation or data.

Check the quantity that matters

Measure solution error, boundary behaviour and relevant conserved quantities on evaluation cases that were not used to tune the method. If a model predicts a time-dependent state, examine how error accumulates over a rollout rather than reporting only a single step.

Limit: satisfying a penalty approximately is different from satisfying a constraint exactly. Neither establishes general accuracy without further analysis.

Explore the computational physics themes →

Making distribution shift visible

A random split tests performance on data that may be very similar to the training set. Real applications can involve a later time period, a different device or an operating condition absent from the original observations.

Construct evaluation groups around the intended use. Keep records from the same underlying source together when their similarity could leak information across splits. Fit normalisation and feature selection only on the training portion, then apply those choices to the held-out data.

Report more than an average

Show error by relevant operating condition and report the number of examples in each group. Inspect rare but consequential failures. Where probabilistic outputs are used, evaluate calibration on the target setting as well as predictive accuracy.

Limit: a finite suite of shift tests covers the scenarios it contains. It cannot certify reliability under every future change. Define the conditions that would require renewed evaluation.

Use the industry workshop checklist →