Coursebooks

Advanced Topics in Information Theory

COM-621

Lecturer(s) :

Gastpar Michael Christoph

Language:

English

Frequency

Only this year

Remark

Next time : Spring 2021

Summary

The class will focus on information-theoretic progress of the last decade. Topics include: Network Information Theory ; Information Measures: definitions, properties, and applications to probabilistic models.

Content

1) Information measures: Definitions, properties, applications, pitfalls.
- Mutual information
- Directed information
- Wyner's common information
- Entropy-power inequality
- Renyi- and f-divergences
- Extremization of information measures 

2) Information measures in probabilistic systems
- Generalization guarantees for learning algorithms
- Compressed Sensing

3) Network Information Theory
- Classical channel settings: Multiple-Access, Broadcast, Relay
- Classical source settings: Slepian-Wolf, Lossy source coding, CEO problem
- "Gaussian location" model and problem
- Caching
- Application to federated learning?

Learning Prerequisites

Recommended courses

COM-404 Information Theory and Coding

Learning Outcomes

By the end of the course, the student must be able to:

Assessment methods

Students will work on projects.

Resources

Bibliography

Cover and Thomas, Elements of Information Theory (2nd ed), Wiley, 2006.

El Gamal and Kim, Network Information Theory, Cambridge, 2011.

Ressources en bibliothèque

In the programs

Reference week

 
      Lecture
      Exercise, TP
      Project, other

legend

  • Autumn semester
  • Winter sessions
  • Spring semester
  • Summer sessions
  • Lecture in French
  • Lecture in English
  • Lecture in German