Learning theory

CS-526 / 6 credits

Teacher: Macris Nicolas

Language: English

Summary

Machine learning and data analysis are becoming increasingly central in many sciences and applications. This course concentrates on the theoretical underpinnings of machine learning.

Content

  • Basics : statistical learning framework, Probably Approximately Correct (PAC) learning, learning with a finite number of classes, Vapnik-Chervonenkis (VC).
  • Bias-variance tradeoff and modern double descent phenomena.
  • Neural Nets : representation power of neural nets.
  • Stochastic gradient descent, modern aspects: mean field approach, neural tangent kernel.
  • Matrix factorization, Tensor decompositions and factorization, Jenrich's tehorem, Alternating least squares, Tucker decompositions.
  • Applications: e.g. Learning mixture models, topic modeling.

Learning Prerequisites

Recommended courses

  • Analysis I, II, III
  • Linear Algebra
  • Machine learning
  • Probability
  • Algorithms (CS-250)

Learning Outcomes

By the end of the course, the student must be able to:

  • Explain the framework of PAC learning
  • Explain the importance basic concepts such as VC dimension, bias-variance tradeoff and double descent
  • Describe basic facts about representation of functions by neural networks
  • Describe recent results on specific topics e.g., matrix and tensor factorization, learning mixture models

Teaching methods

  • Lectures
  • Exercises

Expected student activities

  • Attend lectures
  • Attend exercises sessions and do the homework

Assessment methods

Final exam and graded homeworks

Supervision

Office hours Yes
Assistants Yes
Forum Yes
Others Course website

In the programs

  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Exam form: Written (summer session)
  • Subject examined: Learning theory
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional

Reference week

Related courses

Results from graphsearch.epfl.ch.