Fundamentals of inference and learning

EE-411 / 4 credits

Teacher: Krzakala Florent Gérard

Language: English

Summary

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and practical computational aspects in python.

Content

The topics will be chosen from the following basic outline:

  • Statistical inference: Estimators, Bias-Variance, Consistency, Efficiency, Maximum likelihood, Fisher Information.
  • Bayesian inference, Priors, A posteriori estimation, Expectation-Minimization.
  • Supervised learning : Linear Regression, Ridge, Lasso, Sparse problems, high-dimensional Data, Kernel methods, Boosting, Bagging. K-NN, Support Vector Machines, logistic regression, Optimal Margin Classifier
  • Statistical learning theory: VC Bounds and Uniform convergence, Implicit regularisation, Double-descent
  • Unsupervised learning : Mixture Models, PCA & Kernel PCA, k-means
  • Deep learning: multi-layer nets, convnets, auto-encoder, Gradient-descent algorithms
  • Basics of Generative models & Reinforcement learning

 

Keywords

Statisitics, Supervised and unsupervised learning

Learning Prerequisites

Required courses

* Basic probability theory.

* Basic knowlegde of python programing

* Basic linear algebra, and calculus

 

 

 

 

 

 

Recommended courses

Probability and statistics.

Advance Python

Basic optimization

 

Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra, and be familiar with python.

 

 

 

Learning Outcomes

By the end of the course, the student must be able to:

  • Formulate statistical models and apply them to statistical learning
  • Apply machine learning technics to data science problems
  • Solve concrete data science problems
  • Explain and understand the fundamental principle of learning theory

Assessment methods

* Homeworks during the lectures

* Final project,

Resources

Moodle Link

Videos

In the programs

  • Semester: Fall
  • Exam form: During the semester (winter session)
  • Subject examined: Fundamentals of inference and learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Project: 2 Hour(s) per week x 14 weeks
  • Type: mandatory
  • Semester: Fall
  • Exam form: During the semester (winter session)
  • Subject examined: Fundamentals of inference and learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Project: 2 Hour(s) per week x 14 weeks
  • Type: mandatory

Reference week

Related courses

Results from graphsearch.epfl.ch.