Empirical processes

Summary

We learn how to control the nonasymptotic and random behavior of collections of estimators, when indexed by classes of functions/sets. Examples range from prototypical estimators used by statisticians, to nonparametric models used in machine learning.

Content

Keywords

Empirical processes, concentration of measure, probabilistic inequalities, generalization properties, learning algorthims.

Learning Prerequisites

Important concepts to start the course

Previous courseworks in analysis, mathematical statistics and probability are highly required, with interests in statistical learning theory and algorithms.

Learning Outcomes

By the end of the course, the student must be able to:

• Formulate the fundamental framework related to empirical processes
• Manipulate probabilistic inequalities to empirical estimators
• Describe the concept of concentration-of-measure phenomenon for empirical processes
• Apply concentration inequalities to derive the performance of statistical learning procedures

Teaching methods

The semester will be composed of:

• Blackboard lectures.
• Exercise sessions.

Expected student activities

Student are expected to:

• Attend to all lectures.
• Participate to exercise weekly sessions.

Assessment methods

Written final exam; mini project.

Supervision

Resources

Bibliography

• Stéphane Boucheron, Gábor Lugosi, and Pascal Massart, Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, Oxford Academic, 2013.

• Víctor H. Peña , Evarist Giné, Decoupling. From Dependence to Independence. Probability and Its Applications (PIA), Springer New York, 1999.

• Martin J. Wainwright. High-Dimensional Statistics, A Non-Asymptotic Viewpoint. Cambridge University Press, 2019.

Notes/Handbook

Yes

Moodle Link

• https://go.epfl.ch/MATH-522