Deep learning
EE-559 / 4 credits
Teacher:
Language: English
Remark: pas donné en 2022-23
Summary
The objective of this course is to provide a complete introduction to deep machine learning. How to design a neural network, how to train it, and what are the modern techniques that specifically handle very large networks.
Content
The course aims at providing an overview of existing processings and methods, at teaching how to design and train a deep neural network for a given task, and at providing the theoretical basis to go beyond the topics directly seen in the course.
It will touch on the following topics:
- What is deep learning, introduction to tensors.
- Basic machine-learning, empirical risk minimization, simple embeddings.
- Linear separability, multi-layer perceptrons, back-propagation.
- Generalized networks, autograd, batch processing, convolutional networks.
- Initialization, optimization, and regularization. Drop-out, batchnorm, resnets.
- Deep models for Computer Vision.
- Analysis of deep models.
- Auto-encoders, embeddings, and generative models.
- Recurrent and attention models, Natural Language Processing.
Concepts will be illustrated with examples in the PyTorch framework (http://pytorch.org).
Keywords
machine learning, neural networks, deep learning, computer vision, python, pytorch
Learning Prerequisites
Required courses
- Linear algebra (vector, matrix operations, Euclidean spaces).
- Differential calculus (Jacobian, Hessian, chain rule).
- Python programming.
- Basics in probabilities and statistics (discrete and continuous distributions, normal density, law of large numbers, conditional probabilities, Bayes, PCA)
Recommended courses
- Basics in optimization (notion of minima, gradient descent).
- Basics in algorithmic (computational costs).
- Basics in signal processing (Fourier transform, wavelets).
Teaching methods
Ex-cathedra with exercise sessions and mini-projects. Possibly invited speakers.
Assessment methods
Mini-projects by groups of students, and one final written exam.
Resources
Notes/Handbook
Not mandatory: http://www.deeplearningbook.org/
Websites
In the programs
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Exam form: Written (summer session)
- Subject examined: Deep learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
Reference week
| Mo | Tu | We | Th | Fr | |
| 8-9 | |||||
| 9-10 | |||||
| 10-11 | |||||
| 11-12 | |||||
| 12-13 | |||||
| 13-14 | |||||
| 14-15 | |||||
| 15-16 | |||||
| 16-17 | |||||
| 17-18 | |||||
| 18-19 | |||||
| 19-20 | |||||
| 20-21 | |||||
| 21-22 |
Légendes:
Lecture
Exercise, TP
Project, other