Learning theory
Summary
Machine learning and data analysis are becoming increasingly central in many sciences and applications. This course concentrates on the theoretical underpinnings of machine learning.
Content
- Basics : statistical learning framework, Probably Approximately Correct (PAC) learning, learning with a finite number of classes, Vapnik-Chervonenkis (VC).
- Bias-variance tradeoff and modern double descent phenomena.
- Stochastic gradient descent, modern aspects: mean field approach, neural tangent kernel.
- Diffusion methods.
- Tensor decompositions and factorization, Jenrich's tehorem, Alternating least squares, Tucker decompositions. Applications: e.g. Learning mixture models, topic modeling.
Learning Prerequisites
Recommended courses
- Analysis I, II, III
- Linear Algebra
- Machine learning
- Probability
- Algorithms (CS-250)
Learning Outcomes
By the end of the course, the student must be able to:
- Explain the framework of PAC learning
- Explain the importance basic concepts such as VC dimension and non-uniform learnability
- Describe basic facts about representation of functions by neural networks
- Describe recent results on specific topics e.g., graphical mdoel learning, matrix and tensor factorization, learning mixture models
- Explain the importance basic concepts such as VC dimension, bias-variance tradeoff and double descent
- Describe recent results on specific topics e.g., matrix and tensor factorization, learning mixture models
Teaching methods
- Lectures
- Exercises
Expected student activities
- Attend lectures
- Attend exercises sessions and do the homework
Assessment methods
Final exam and graded homeworks
Supervision
| Office hours | Yes |
| Assistants | Yes |
| Forum | Yes |
| Others | Course website |
In the programs
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Learning theory
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional