Coursebooks 2017-2018

PDF
 

Optimization for machine learning

CS-439

Lecturer(s) :

Jaggi Martin

Language:

English

Summary

This course teaches an overview of modern optimization methods, for applications in machine learning and data science. In particular, scalability of algorithms to large datasets will be discussed in theory and in implementation.

Content

This course teaches an overview of modern optimization methods, for applications in machine learning and data science. In particular, scalability of algorithms to large datasets will be discussed in theory and in implementation.

Basic Contents:

Convexity, Gradient Methods, Proximal algorithms, Stochastic and Online Variants of mentioned methods, Coordinate Descent Methods, Subgradient Methods, Frank-Wolfe, Accelerated Methods, Primal-Dual context and certificates, Lagrange and Fenchel Duality, Second-Order Methods, Quasi-Newton Methods. Black-Box Optimization.

Advanced Contents:

Parallel and Distributed Optimization Algorithms, Synchronous and Asynchronous Communication.

Computational and Statistical Trade-Offs (Time vs Data vs Accuracy). Variance Reduced Methods, and Lower Bounds.

Non-Convex Optimization: Convergence to Critical Points, Saddle-Point methods, Alternating minimization for matrix and tensor factorizations

Keywords

Optimization, Machine learning

Learning Prerequisites

Recommended courses

Important concepts to start the course

Learning Outcomes

By the end of the course, the student must be able to:

Transversal skills

Teaching methods

Expected student activities

Students are expected to:

Assessment methods

Supervision

Office hours Yes
Assistants Yes
Forum Yes

Resources

Virtual desktop infrastructure (VDI)

No

Websites

In the programs

  • Data Science, 2017-2018, Master semester 2
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks
  • Computer Science, 2017-2018, Master semester 2
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks
  • Computational science and Engineering, 2017-2018, Master semester 2
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks
  • Computational science and Engineering, 2017-2018, Master semester 4
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks
  • Communication Systems - master program, 2017-2018, Master semester 2
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks
  • Communication Systems - master program, 2017-2018, Master semester 4
    • Semester
      Spring
    • Exam form
      Written
    • Credits
      4
    • Subject examined
      Optimization for machine learning
    • Lecture
      2 Hour(s) per week x 14 weeks
    • Exercises
      2 Hour(s) per week x 14 weeks

Reference week

MoTuWeThFr
8-9
9-10
10-11
11-12
12-13
13-14 BC01
14-15
15-16 BC01
16-17
17-18
18-19
19-20
20-21
21-22
Lecture
Exercise, TP
Project, other

legend

  • Autumn semester
  • Winter sessions
  • Spring semester
  • Summer sessions
  • Lecture in French
  • Lecture in English
  • Lecture in German