Machine learning
Summary
Machine learning methods are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and practically implemented.
Content
- Basic regression and classification concepts and methods: Linear models, overfitting, linear regression, Ridge regression, logistic regression, k-NN, SVMs and kernel methods
- Fundamental concepts: cost-functions and optimization, cross-validation and bias-variance trade-off, curse of dimensionality.
- Neural Networks: Representation power, backpropagation, activation functions, CNN, regularization, data augmentation, dropout
- Unsupervised learning: k-means clustering, gaussian mixture models and the EM algorithm. Basics of self-supervised learning
- Dimensionality reduction: PCA and matrix factorization, word embeddings
- Advanced methods: Adversarial learning, Generative adversarial networks
Keywords
- Machine learning, pattern recognition, deep learning, neural networks, data mining, knowledge discovery, algorithms
Learning Prerequisites
Required courses
- Analysis I, II, III
- Linear Algebra
- Probability and Statistics (MATH-232)
- Algorithms I (CS-250)
Recommended courses
- Introduction to machine learning (CS-233)
- ...or similar bachelor lecture from other sections
Important concepts to start the course
- Basic probability and statistics (conditional and joint distribution, independence, Bayes rule, random variables, expectation, mean, median, mode, central limit theorem)
- Basic linear algebra (matrix/vector multiplications, systems of linear equations, SVD)
- Multivariate calculus (derivative w.r.t. vector and matrix variables)
- Basic Programming Skills (labs will use Python)
Learning Outcomes
By the end of the course, the student must be able to:
- Define the following basic machine learning problems: Regression, classification, clustering, dimensionality reduction, time-series
- Explain the main differences between them
- Implement algorithms for these machine learning models
- Optimize the main trade-offs such as overfitting, and computational cost vs accuracy
- Implement machine learning methods to real-world problems, and rigorously evaluate their performance using cross-validation. Experience common pitfalls and how to overcome them
- Explain and understand the fundamental theory presented for ML methods
- Conduct a real-world interdisciplinary machine learning project, in collaboration with application domain experts
- Define the following basic machine learning models: Regression, classification, clustering, dimensionality reduction, neural networks, time-series analysis
Teaching methods
- Lectures
- Lab sessions
- Course Projects
Expected student activities
Students are expected to:
- attend lectures
- attend lab sessions and work on the weekly theory and coding exercises
- work on projects using the code developed during labs, in small groups
Assessment methods
- Written final exam
- Continuous control (Course projects)
Supervision
| Office hours | Yes |
| Assistants | Yes |
| Forum | Yes |
Resources
Virtual desktop infrastructure (VDI)
No
Bibliography
- Christopher Bishop, Pattern Recognition and Machine Learning
- Kevin Murphy, Machine Learning: A Probabilistic Perspective
- Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning
- Michael Nielsen, Neural Networks and Deep Learning
- (Jerome Friedman, Robert Tibshirani, Trevor Hastie, The elements of statistical learning : data mining, inference, and prediction)
Ressources en bibliothèque
- The elements of statistical learning : data mining, inference, and prediction / Friedman
- Pattern Recognition and Machine Learning / Bishop
- Understanding Machine Learning / Shalev-Shwartz
- Machine Learning / Murphy
Références suggérées par la bibliothèque
Notes/Handbook
https://github.com/epfml/ML_course
Websites
Moodle Link
In the programs
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Fall
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
- Exam form: Written (winter session)
- Subject examined: Machine learning
- Lecture: 4 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: optional
Reference week
| Mo | Tu | We | Th | Fr | |
| 8-9 | |||||
| 9-10 | |||||
| 10-11 | RLC E1 240 | ||||
| 11-12 | |||||
| 12-13 | |||||
| 13-14 | |||||
| 14-15 | INF1 INF119 INM202 INJ218 INR219 | ||||
| 15-16 | |||||
| 16-17 | RLC E1 240 | ||||
| 17-18 | |||||
| 18-19 | |||||
| 19-20 | |||||
| 20-21 | |||||
| 21-22 |
Légendes:
Lecture
Exercise, TP
Project, other
Tuesday, 16h - 18h: Lecture RLC E1 240
Wednesday, 10h - 12h: Lecture RLC E1 240
Thursday, 14h - 16h: Exercise, TP INF1
INF119
INM202
INJ218
INR219