Dynamic programming and optimal control

Next course will take place from October 1 to 2 and 5 to 7, 2026.

Day 1
Lecture 1: Intro and Course Outline
Lecture 2: Mathematical Modeling Framework (Markov Decision Processes)
Lecture 3: The Principle of Optimality and Dynamic Programming (DP)
Day 2-3
Finite Space Systems
Lecture 4: Markov Chains
Lecture 5: DP for finite space Markov Decision Processes
Lecture 6: Coding DP (inventory and river flow examples)
Lecture 7: Infinite Horizon Problems, Value Iteration, Policy Iteration
Day 4-5
Continuous Space Systems and Linear Quadric Problems (LQ)
Lecture 8: Dynamic Programming in continuous space problems, curse of dimensionality, limitations
Lecture 9: DP in LQ problems
Lecture 10: LQ variations, time-varying, infinite horizon, multiplicative noise, dynamic games, etc.
Day 5-6
Approximate DP and Reinforcement Learning (RL) and Advanced Topics
Lecture 11: Approximate Dynamic Programming I (touch on MPC, RL)
Lecture 12: Approximate Dynamic Programming II (touch on MPC, RL)
Lecture 13: Supply Chain Example, Project Description
Lecture 14: Imperfect State Information (time permitting)