Mathematics of machine learning

MATH-520 / 5 crédits

Langue: Anglais

Summary

Mathematical analysis of modern supervised machine learning techniques, from linear methods to artificial neural networks.

Introduction (supervised learning, risk, error decomposition, over-fitting and capacity control + cross-validation, Bayes predictor for classification and regression)
Differentiable programming (backpropagation algorithm) and theoretical challenges posed by modern methods (large deep neural networks)
Statistical analysis of Empirical Risk Minimization (learning theory, from finite number of hypotheses to Rademacher / covering numbers)
First-order methods for optimization (gradient descent, stochastic gradient descent).
Kernel methods (positive-definite kernels and Reproducing Kernel Hilbert Spaces)
Algorithmic regularization of gradient descent (reparameterized models, least-squares, mirror descent, logistic loss and max-margin)
Dynamics of wide neural networks (parameterizations, neural tangent kernel and feature-learning limits)
Statistical analysis of interpolating methods (double descent, benign overfitting)

Supervised learning, Machine learning, Neural networks, Optimization, Statistics

Analysis, Linear Algebra, Probability and Statistics

A good knowledge of undergraduate mathematics is important.
Ability to code in a scientific computing programming language of your choice (e.g. Python, Matlab, Julia). The course will involve coding exercises and assignments.
Having followed an introductory class on machine learning is beneficial.

Contextualise the research literature on theoretical machine learning
Interpret the practical behavior of complex machine learning pipelines through the lens of mathematical theory
Implement simple supervised learning algorithms from scratch
Reason on how statistical and optimization phenomena interact in machine learning practice
Distinguish between what is known and what is not known in the theory of deep learning

Homeworks, projects, presentation

Légendes:

Cours

Exercice, TP

Projet, autre