COM-621 / 2 credits

Teacher(s): Gastpar Michael Christoph, Issa Ibrahim

Language: English

Remark: Not given this year, next time: Spring 2025


Frequency

Only this year

Summary

The class will focus on information-theoretic progress of the last decade. Topics include: Network Information Theory ; Information Measures: definitions, properties, and applications to probabilistic models.

Content

1) Information measures: Definitions, properties, applications, pitfalls.
- Mutual information
- Directed information
- Wyner's common information
- Entropy-power inequality
- Renyi- and f-divergences
- Extremization of information measures 

2) Information measures in probabilistic systems
- Generalization guarantees for learning algorithms
- Compressed Sensing

3) Network Information Theory
- Classical channel settings: Multiple-Access, Broadcast, Relay
- Classical source settings: Slepian-Wolf, Lossy source coding, CEO problem
- "Gaussian location" model and problem
- Caching
- Application to federated learning?

Learning Prerequisites

Recommended courses

COM-404 Information Theory and Coding

Learning Outcomes

By the end of the course, the student must be able to:

  • Generalize information measures
  • Formulate estimation, inference and decision problems via the lens of information measures
  • Analyze communication networks via information measures
  • Manipulate information measures

Assessment methods

Students will work on projects.

Resources

Bibliography

Cover and Thomas, Elements of Information Theory (2nd ed), Wiley, 2006.

El Gamal and Kim, Network Information Theory, Cambridge, 2011.

Ressources en bibliothèque

Moodle Link

In the programs

  • Exam form: Project report (session free)
  • Subject examined: Advanced Topics in Information Theory
  • Lecture: 14 Hour(s)
  • Exercises: 14 Hour(s)
  • Type: optional

Reference week

Related courses

Results from graphsearch.epfl.ch.