MATH-520 / 5 crédits

Enseignant:

Langue: Anglais

Remark: Pas donné en 2024-25


Summary

Mathematical analysis of modern supervised machine learning techniques, from linear methods to artificial neural networks.

Content

  • Introduction (supervised learning, risk, error decomposition, over-fitting and capacity control + cross-validation, Bayes predictor for classification and regression)

  • Differentiable programming (backpropagation algorithm) and theoretical challenges posed by modern methods (large deep neural networks)

  • Statistical analysis of Empirical Risk Minimization (learning theory, from finite number of hypotheses to Rademacher / covering numbers)

  • First-order methods for optimization (gradient descent, stochastic gradient descent).

  • Kernel methods (positive-definite kernels and Reproducing Kernel Hilbert Spaces)

  • Algorithmic regularization of gradient descent (reparameterized models, least-squares, mirror descent, logistic loss and max-margin)

  • Dynamics of wide neural networks (parameterizations, neural tangent kernel and feature-learning limits)

 

Keywords

Supervised learning, Machine learning, Neural networks, Optimization, Statistics

Learning Prerequisites

Required courses

Analysis, Linear Algebra, Probability and Statistics

Important concepts to start the course

  • A good knowledge of undergraduate mathematics is important.

  • Ability to code in a scientific computing programming language of your choice (e.g. Python, Matlab, Julia). The course will involve coding exercises and assignments.

  • Having followed an introductory class on machine learning is beneficial.

 

Learning Outcomes

By the end of the course, the student must be able to:

  • Contextualise the research literature on theoretical machine learning
  • Interpret he practical behavior of complex machine learning pipelines through the lens of mathematical theory
  • Implement simple supervised learning algorithms from scratch
  • Reason on how statistical and optimization phenomena interact in machine learning practice
  • Distinguish between what is known and what is not known in the theory of deep learning

Assessment methods

Homeworks, projects, presentation

Resources

Moodle Link

Dans les plans d'études

  • Semestre: Automne
  • Forme de l'examen: Pendant le semestre (session d'hiver)
  • Matière examinée: Topics in machine learning
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel
  • Semestre: Automne
  • Forme de l'examen: Pendant le semestre (session d'hiver)
  • Matière examinée: Topics in machine learning
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel
  • Semestre: Automne
  • Forme de l'examen: Pendant le semestre (session d'hiver)
  • Matière examinée: Topics in machine learning
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel
  • Semestre: Automne
  • Forme de l'examen: Pendant le semestre (session d'hiver)
  • Matière examinée: Topics in machine learning
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel

Semaine de référence

Cours connexes

Résultats de graphsearch.epfl.ch.