Advanced Topics in Information Theory
COM-621 / 2 credits
Teacher(s): Gastpar Michael Christoph, Issa Ibrahim
Language: English
Remark: Not given this year, next time: Spring 2025
Frequency
Only this year
Summary
The class will focus on information-theoretic progress of the last decade. Topics include: Network Information Theory ; Information Measures: definitions, properties, and applications to probabilistic models.
Content
1) Information measures: Definitions, properties, applications, pitfalls.
- Mutual information
- Directed information
- Wyner's common information
- Entropy-power inequality
- Renyi- and f-divergences
- Extremization of information measures
2) Information measures in probabilistic systems
- Generalization guarantees for learning algorithms
- Compressed Sensing
3) Network Information Theory
- Classical channel settings: Multiple-Access, Broadcast, Relay
- Classical source settings: Slepian-Wolf, Lossy source coding, CEO problem
- "Gaussian location" model and problem
- Caching
- Application to federated learning?
Learning Prerequisites
Recommended courses
COM-404 Information Theory and Coding
Learning Outcomes
By the end of the course, the student must be able to:
- Generalize information measures
- Formulate estimation, inference and decision problems via the lens of information measures
- Analyze communication networks via information measures
- Manipulate information measures
Assessment methods
Students will work on projects.
Resources
Bibliography
Cover and Thomas, Elements of Information Theory (2nd ed), Wiley, 2006.
El Gamal and Kim, Network Information Theory, Cambridge, 2011.
Ressources en bibliothèque
Moodle Link
In the programs
- Exam form: Project report (session free)
- Subject examined: Advanced Topics in Information Theory
- Lecture: 14 Hour(s)
- Exercises: 14 Hour(s)
- Type: optional