MGT-484 / 4 credits

Teacher: Sutter Tobias

Language: English


Summary

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc.

Content

Keywords

Markov chains, Markov decision processes, dynamic programming, optimal control

Learning Outcomes

By the end of the course, the student must be able to:

  • Understand the concept of a discrete-time Markov chain and know how Markov chains are used to model random phenomena
  • Compute several properties of a given Markov chain, such as hitting probabilities, expected hitting times, invariant distributions and the long-run proportion of time spent in a given state
  • Formalize decision problems under uncertainty as optimal control models
  • Solve optimal control models via dynamic programming
  • Be able to read the technical literature in applied probability and to undertake independent self-study (or research) in the future

Transversal skills

  • Communicate effectively, being understood, including across different languages and cultures.
  • Assess one's own level of skill acquisition, and plan their on-going learning goals.

Teaching methods

Classical formal teaching interlaced with practical exercices.

Expected student activities

Active participation in exercise sessions is essential.

Assessment methods

  • 30% midterm exam (04.11.2022)
  • 70% final exam

Supervision

Office hours Yes
Assistants Yes
Forum No

Resources

Bibliography

Introduction to Probability Models, 10th edition, Sheldon M. Ross, Academic Press, 2009.

Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas, Athena Scientific, 2005.

Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis, Athena Scientific, 2002.

 

 

Ressources en bibliothèque

Prerequisite for

Advanced MTE courses

In the programs

  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks
  • Semester: Fall
  • Exam form: Written (winter session)
  • Subject examined: Applied probability & stochastic processes
  • Lecture: 2 Hour(s) per week x 14 weeks
  • Exercises: 2 Hour(s) per week x 14 weeks

Reference week

 MoTuWeThFr
8-9     
9-10     
10-11    GCB330
11-12    
12-13     
13-14     
14-15    CM1
15-16    
16-17     
17-18     
18-19     
19-20     
20-21     
21-22     

Friday, 10h - 12h: Lecture GCB330

Friday, 14h - 16h: Exercise, TP CM1