Nonlinear optimization
Summary
This course introduces students to continuous, nonlinear optimization. We investigate properties of optimization problems with continuous variables, and we analyze and implement important algorithms to solve constrained and unconstrained problems.
Content
* Unconstrained optimization of differentiable functions
-- Necessary optimality conditions
-- The role of Lipschitz assumptions
-- Gradient descent and Newton's method
-- The trust-regions method
* Constrained optimization of differentiable functions
-- Necessary optimality conditions, cones
-- Notions of duality
-- The quadratic penalty method
-- The augmented Lagrangian method
* Special topics (to be determined; e.g.: convexity, relaxations, conic programming, nonsmooth problems and smoothing, nonlinear least-squares, quasi-Newton methods, derivative free methods, ...)
Note: as this is a fairly new course, the precise contents may change during the semester.
Learning Prerequisites
Required courses
Students are expected to be comfortable with linear algebra, analysis and mathematical proofs.
Students are expected to be comfortable writing simple code in Matlab. They may be allowed to write some of their work in Python or Julia upon request.
Learning Outcomes
By the end of the course, the student must be able to:
- Recognize and formulate a mathematical optimization problem.
- Analyze and implement the gradient descent method, Newton's method, the trust-region method and the augmented Lagrangian method, among others.
- Establish and discuss local and global convergence guarantees for iterative algorithms.
- Exploit elementary notions of convexity and duality in optimization.
- Apply the general theory to particular cases.
- Prove some of the most important theorems studied in class.
Teaching methods
Lectures + exercise sessions
Expected student activities
Students are expected to attend lectures and participate actively in class and exercises. Exercises will include both theoretical work and programming assignments. Students also complete projects that likewise include theoretical and numerical work.
Assessment methods
Final exam (40%) + homework/projects (60%)
Supervision
| Office hours | No |
| Assistants | Yes |
| Forum | No |
Resources
Bibliography
Book "Numerical Optimization", J. Nocedal and S. Wright, Springer 2006: https://link.springer.com/book/10.1007/978-0-387-40065-5
Ressources en bibliothèque
Notes/Handbook
Lecture notes provided by the lecturer.
In the programs
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Nonlinear optimization
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks