Statistical machine learning

MATH-412 / 5 credits

Teacher(s): Obozinski Guillaume Romain, Zemel Yoav

Language: English

Summary

A course on statistical machine learning for supervised and unsupervised learning

Content

  • Introduction: supervised and unsupervised learning, loss functions, train and test errors, bias-variance tradeoff, model complexity and overfitting, linear regression, k-nearest neighbors.
  • Regression: linear regression, model selection, ridge and Lasso.
  • Local averaging methods: k-nearest neighbor, Nadaraya-Waston estimators
  • Simple validation, cross-validation and leave-one-out.
  • Classification: linear discriminant analysis, logistic regression.
  • Nonparametric regression: smoothing splines, reproducing kernel Hilbert spaces.
  • Support vector machines (possibly) and kernelized linear regression.
  • Tree-based methods: classification and regression trees, bagging, random forests.
  • Boosting: AdaBoost, boosted trees.
  • Deep learning: introduction to neural networks (architecture, backpropagation, CNNs)
  • Unsupervised learning: principal component analysis, k-means, Gaussian mixtures and the EM algorithm.

Learning Prerequisites

Important concepts to start the course

This course introduces machine learning with a statistical  and mathematical perspective. Prior to following this course, the student must have very good knowledge of basic probabilty and statistics (statistical modeling and inference, linear regression). Some of the concepts used in course: abstract conditional expectations, multivariate linear regression, convexity, strong convexity, vector space, Hilbert space.

Learning Outcomes

By the end of the course, the student must be able to:

  • Assess / Evaluate underfitting / overfitting of ML algorithms

Transversal skills

  • Write a scientific or technical report.
  • Make an oral presentation.
  • Demonstrate the capacity for critical thinking
  • Take feedback (critique) and respond in an appropriate manner.
  • Demonstrate a capacity for creativity.

Teaching methods

Ex cathedra lectures, exercises and computer practicals in the classroom and at home, and a project in a group of 3 students.

Assessment methods

Written final exam (70%) + Project of implementation or application on real data of a model/algorithm based on a classical research paper describing an important method from the literature. (30%)

Resources

Moodle Link

In the programs

  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Exam form: Written (summer session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Exam form: Written (summer session)
  • Subject examined: Statistical machine learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional

Reference week

Related courses

Results from graphsearch.epfl.ch.