Fiches de cours

Mathematics of data: from theory to computation

EE-556

Lecturer(s) :

Cevher Volkan

Language:

English

Summary

This course reviews recent advances in convex optimization and statistical analysis in the wake of Big Data. We provide an overview of the emerging convex formulations and their guarantees, describe scalable solution techniques, and illustrate the role of parallel and distributed computation.

Content

The course consists of the following topics

Lecture 1: Introduction to convex optimization and iterative methods.

Lecture 2: Review of basic probability theory. Maximum likelihood, M-estimators, and empirical risk minimization as a motivation for convex optimization.

Lecture 3: Fundamental concepts in convex analysis. Review of linear algebra.

Lecture 4: Unconstrained smooth minimization I: Concept of an iterative optimization algorithm. Convergence rates. Characterization of functions.

Lecture 5: Unconstrained smooth minimization II: Gradient and accelerated gradient methods.

Lecture 6: Unconstrained smooth minimization III: The quadratic case. The conjugate gradient method. Variable metric algorithms.

Lecture 7: Stochastic gradient methods.

Lecture 8: Non-convex optimization and deep learning: Non0cnovex optimization. Neural Networks. Adaptive gradient methods.

Lecture 9: Composite convex minimization I: Subgradient method. Proximal and accelerated proximal gradient methods.

Lecture 10: Composite convex minimization II: Proximal Newton-type methods. Stochastic proximal gradient methods.

Lecture 11: Constrained convex minimization I: The primal-dual approach. Smoothing approaches for non-smooth convex minimization.

Lecture 12: Constrained convex minimization II: The Frank-Wolfe method. The universal primal-dual gradient method. The alternating direction method of multipliers (ADMM).

Lecture 13: Constrained Convex optimization III.

Learning Prerequisites

Required courses

Previous coursework in calculus, linear algebra, and probability is required. Familiarity with optimization is useful.

Learning Outcomes

By the end of the course, the student must be able to:

In the programs

Reference week

 MoTuWeThFr
8-9     
9-10BC01    
10-11    
11-12     
12-13     
13-14     
14-15     
15-16     
16-17    BC01
BC07-08
CE1103
17-18    
18-19     
19-20     
20-21     
21-22     
 
      Lecture
      Exercise, TP
      Project, other

legend

  • Autumn semester
  • Winter sessions
  • Spring semester
  • Summer sessions
  • Lecture in French
  • Lecture in English
  • Lecture in German