Deep learning

EE-559 / 4 credits

Teacher: Cavallaro Andrea

Language: English

Withdrawal: It is not allowed to withdraw from this subject after the registration deadline.

Summary

This course explores how to design reliable discriminative and generative neural networks, the ethics of data acquisition and model deployment, as well as modern multi-modal models.

Content

This course equips students with a comprehensive foundation of modern deep learning, enabling students to design and train discriminative and generative neural networks for a wide range of tasks. Topics include:

  • Deep learning applications (natural language processing, computer vision, audio processing, biology, robotics, science), principles and regulations
  • Loss functions, data and labels, data provenance
  • Training models: gradients and initialization
  • Generalization and performance
  • Transformers
  • Graph neural networks
  • Generative adversarial networks
  • Variational autoencoders
  • Diffusion models
  • Multi-modal models
  • Interpretability, explanations, bias and fairness

Keywords

machine learning, neural networks, deep learning, python

Learning Prerequisites

Required courses

  • Basics in probabilities and statistics
  • Linear algebra
  • Differential calculus
  • Python programming

Recommended courses

  • Basics in optimization
  • Basics in algorithmic
  • Basics in signal processing

Important concepts to start the course

Discrete and continuous distributions, normal density, law of large numbers, conditional probabilities, Bayes, PCA, vector, matrix operations, Euclidean spaces, Jacobian, Hessian, chain rule, notion of minima, gradient descent, computational costs, Fourier transform, convolution.

Learning Outcomes

By the end of the course, the student must be able to:

  • Interpret the performance of a deep learning model
  • Analyze the limitations of a deep learning model
  • Justify the choices for training and testing a deep learning model
  • Propose new solutions for a given application

Transversal skills

  • Respect relevant legal guidelines and ethical codes for the profession.
  • Take account of the social and human dimensions of the engineering profession.
  • Design and present a poster.
  • Make an oral presentation.
  • Demonstrate the capacity for critical thinking

Teaching methods

Ex-cathedra lectures, class discussion, exercises (using python), group project.

Expected student activities

Attendance to lectures, participation in discussions, completing exercises, completing a project, reading written material (scientific papers and books).

Assessment methods

Excercises and group project.

Resources

Références suggérées par la bibliothèque

Notes/Handbook

Not mandatory: http://www.deeplearningbook.org/

Moodle Link

In the programs

  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional
  • Number of places: 150
  • Exam form: During the semester (summer session)
  • Subject examined: Deep learning
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Type: optional

Reference week

Related courses

Results from graphsearch.epfl.ch.