Probabilistic models of modern AI
Summary
This is a course where we will cover various topics related to recent progress in AI. Emphasis will be on understanding conceptual and mathematical principles, and on relating those to experimental data for machine learning models used in practice.
Content
- Basics: Classification vs Regression, Information Theory
- Gaussian Models and Kernel Methods
- Neural Network Optimization and Neural Tangent Kernel
- Compression and AI
- Loss Functions and Scoring Rules
- LLMs: Basic Models and Transformers
- LLMs: Measures and Sampling, Arrows of Time
- LLMs: Fine-Tuning and Reinforcement Learning
- Causality
- Diffusion Models: Diffusion Processes and Basics
- Diffusion Models: Various Formulations
- Diffusion Models: Information-Theoretic Views
- Other Models: Normalizing Flows, GANs
- Capability Measures
Keywords
Neural Networks, LLMs, Diffusion Models
Learning Prerequisites
Required courses
Beyond the standard bachelor classes, nothing is a pre-requisite, however some familiarity with probability (in an applied sense), coding, and machine learning ideas should greatly help.
Assessment methods
Oral
Dans les plans d'études
- Semestre: Automne
- Forme de l'examen: Oral (session d'hiver)
- Matière examinée: Probabilistic models of modern AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Oral (session d'hiver)
- Matière examinée: Probabilistic models of modern AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Oral (session d'hiver)
- Matière examinée: Probabilistic models of modern AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Oral (session d'hiver)
- Matière examinée: Probabilistic models of modern AI
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel