Foundation models and generative AI
Summary
This course covers the principles, architectures, and applications of foundation models and generative AI, including generative methods, tokenization, multi-modal learning, adaptation, prompting, and their use in reasoning, decision-making, and scientific domains.
Content
This course introduces the principles, methods, and applications of foundation models and generative AI, i.e., two areas that increasingly shape the frontiers of machine learning. Foundation models are large-scale models pretrained on broad and diverse data, capable of adapting to a wide range of downstream tasks across modalities. While not all foundation models are generative, many leverage generative modeling techniques, such as autoregressive prediction, variational inference, diffusion processes, and flow-based transformations, to learn expressive representations and enable open-ended generation.
The course begins with the core generative model families, covering their objectives, mathematical foundations, and applications in language, vision, and scientific domains. It then transitions to the design and scaling of foundation models, focusing on tokenization across modalities, architectural choices, and training strategies. Students will explore multi-modal learning, prompting, adaptation (e.g. fine-tuning, test-time training), and the role of reinforcement learning in aligning model behavior with desired outcomes.
Throughout, the course highlights how foundation models and generative AI increasingly intersect, particularly in the development of world models, where learned generative representations support simulation, planning, and decision-making.
Through lectures, assignments, and hands-on exercises, students will gain both theoretical and practical understanding of the foundations, capabilities, and frontiers of generative and foundation models, and their growing impact across scientific and engineering domains.
Keywords
foundation models, generative AI, multi-modal learning
Learning Prerequisites
Required courses
- CS-233 Introduction to machine learning or CS-433 Machine Learning or equivalent course on the basics of machine learning
Recommended courses
- EE-559 Deep Learning or equivalent course on the basics of deep learning
Related courses
- CS-503 Visual intelligence: machines and minds and CS-552 Modern natural language processing
Important concepts to start the course
- Python programming
- Probability and statistics
- Linear algebra
- Machine learning
Learning Outcomes
By the end of the course, the student must be able to:
- Describe and explain core generative modeling techniques and their conceptual role in foundation models.
- Analyze and compare the architectures, tokenization strategies, and training objectives of foundation models across language, vision, and scientific domains.
- Apply suitable foundation models or generative approaches for a given task and justify their use based on model capabilities and data modality.
- Investigate and interpret recent advances in multi-modal learning, prompting, and decision-making with foundation models.
- Critique and synthesize key contributions from current research papers by relating them to concepts and methods covered in the course.
Teaching methods
Lectures, exercise sessions, coding tutorials, and guided paper reading.
Expected student activities
The students are expected to study the provided lecture material, actively participate in lectures and exercise sessions, engage in discussions, and complete homework assignments. Assessment is based on graded exercises and a final exam.
Assessment methods
- Exam (70%)
- Homeworks (30%)
Supervision
| Office hours | Yes |
| Assistants | Yes |
| Forum | Yes |
Dans les plans d'études
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Foundation models and generative AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 1 Heure(s) hebdo x 14 semaines
- Projet: 1 Heure(s) hebdo x 14 semaines
- Type: optionnel