MGT-484 / 4 credits

Teacher: Kiyavash Negar

Language: English

## Summary

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc.

## Keywords

Markov chains, Markov decision processes, dynamic programming, optimal control

## Required courses

A course in basic probability theory

## Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra.

## Learning Outcomes

By the end of the course, the student must be able to:

• Formulate Markov chain models for dynamic uncertain phenomena.
• Formulate Markov decision process models for dynamic decision problems under uncertainty.
• Use these models to structure real decision-making situations.
• Compute relevant performance measures for Markov models.
• Develop an awareness of the manifold uses of probability theory in management science.

## Transversal skills

• Communicate effectively, being understood, including across different languages and cultures.
• Assess one's own level of skill acquisition, and plan their on-going learning goals.

## Teaching methods

Classical formal teaching interlaced with practical exercices.

## Expected student activities

Active participation in exercise sessions is essential.

## Assessment methods

• 20% homework
• 30% midterm project
• 50% final exam

## Supervision

 Office hours Yes Assistants Yes Forum No

## Bibliography

Introduction to Probability Models, 10th edition, Sheldon M. Ross, Academic Press, 2009.

Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas, Athena Scientific, 2005.

Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis, Athena Scientific, 2002.

## In the programs

• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks
• Semester: Fall
• Exam form: Written (winter session)
• Subject examined: Applied probability & stochastic processes
• Lecture: 2 Hour(s) per week x 14 weeks
• Exercises: 2 Hour(s) per week x 14 weeks

## Reference week

 Mo Tu We Th Fr 8-9 9-10 10-11 11-12 12-13 13-14 14-15 15-16 16-17 17-18 18-19 19-20 20-21 21-22