EE-411 / 4 credits

Teacher: Krzakala Florent Gérard

Language: English

## Summary

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and practical computational aspects in python.

## Content

The topics will be chosen from the following basic outline:

• Statistical inference: Estimators, Bias-Variance, Consistency, Efficiency, Maximum likelihood, Fisher Information.
• Bayesian inference, Priors, A posteriori estimation, Expectation-Minimization.
• Supervised learning : Linear Regression, Ridge, Lasso, Sparse problems, high-dimensional Data, Kernel methods, Boosting, Bagging. K-NN, Support Vector Machines, logistic regression, Optimal Margin Classifier
• Statistical learning theory: VC Bounds and Uniform convergence, Implicit regularisation, Double-descent
• Unsupervised learning : Mixture Models, PCA & Kernel PCA, k-means
• Deep learning: multi-layer nets, convnets, auto-encoder, Gradient-descent algorithms
• Basics of Generative models & Reinforcement learning

## Keywords

Statisitics, Supervised and unsupervised learning

## Required courses

* Basic probability theory.

* Basic knowlegde of python programing

* Basic linear algebra, and calculus

## Recommended courses

Probability and statistics.

Basic optimization

## Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra, and be familiar with python.

## Learning Outcomes

By the end of the course, the student must be able to:

• Formulate statistical models and apply them to statistical learning
• Apply machine learning technics to data science problems
• Solve concrete data science problems
• Explain and understand the fundamental principle of learning theory

## Assessment methods

* Homeworks during the lectures

* Final project,

## In the programs

• Semester: Fall
• Exam form: During the semester (winter session)
• Subject examined: Fundamentals of inference and learning
• Lecture: 2 Hour(s) per week x 14 weeks
• Project: 2 Hour(s) per week x 14 weeks
• Type: mandatory
• Semester: Fall
• Exam form: During the semester (winter session)
• Subject examined: Fundamentals of inference and learning
• Lecture: 2 Hour(s) per week x 14 weeks
• Project: 2 Hour(s) per week x 14 weeks
• Type: mandatory

## Reference week

Tuesday, 10h - 12h: Lecture GRB330

Wednesday, 8h - 10h: Project, other GRB330

## Related courses

Results from graphsearch.epfl.ch.