Fiches de cours 2017-2018

PDF
 

Advanced Topics in Machine Learning

EE-620

Lecturer(s) :

Cevher Volkan

Language:

English

Frequency

Every 2 years

Remarque

Every 2 years. Next time: Spring 2018

Summary

This course describes theory and methods to address three key challenges in data sciences: estimation, prediction, and computation. We use convex analysis and methods as a common connecting theme, and illustrate the main ideas on concrete applications from machine learning and signal processing.

Content

Lecture 1

Overview of the course. Learning-based compressive subsampling. Introduction to submodularity, examples, and properties.

Lecture 2

Discrete optimization. Submodular function maximization. The greedy algorithm. Relations between submodularity and convexity. The Lovazs extension.

Lecture 3

Introduction to structured sparsity. Convex relaxation by biconjugation. Structured sparsity via submodular functions.

Lecture 4

Integer and linear programming. Structured sparsity via totally unimodular constraints. Robust submodular function maximization. The Saturate algorithm.

Lecture 5

Stochastic optimization. Stochastic subgradient method. Stochastic proximal method. Stochastic accelerated methods.

Lecture 6

Variance reduction. Coordinate descent methods for smooth objectives.

Lecture 7

Coordinate descent methods for composite functions. Coordinate descent primal-dual algorithms. Randomized Linear Algebra. Randomized matrix decompositions. Comparison to classical methods.

Lecture 8

Randomized Linear Algebra. Row extraction method. Power method. Column selection methods. Stochastic quasi-Newton Method.

Lecture 9

Introduction to statistical learning theory. Statistics vs statistical learning. Approximation error. Excess risk, Hoeffding's inequality.

Lecture 10

Concentration of measure inequalities. Chernoff-type bounds. Azuma-Hoeffding inequality. Bounded differences inequality.

Lecture 11

Uniform Convergences in Statistical Learning Theory. Classical VC Theory for Binary Classification.

Lecture 12        

Project presentations.'''''

Keywords

Optimization, machine learning, signal processing, statistics.

Learning Prerequisites

Required courses

Multivariable calculus, linear algebra.

Recommended courses

Probability theory.

Learning Outcomes

By the end of the course, the student must be able to:

Assessment methods

Multiple.

Resources

Websites

In the programs

Reference week

 
      Lecture
      Exercise, TP
      Project, other

legend

  • Autumn semester
  • Winter sessions
  • Spring semester
  • Summer sessions
  • Lecture in French
  • Lecture in English
  • Lecture in German