- français
- English
Coursebooks
Applied probability & stochastic processes
MGT-484
Lecturer(s) :
Kiyavash NegarLanguage:
English
Summary
This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc.Content
The following topics will tentatively be covered in the course:
1. Discrete-time Markov chains
- Basic definitions, transition probabilities
- Classification of states
- Stationary and limiting distributions, convergence to equilibrium
- Hitting times and absorption probabilities
- Strong Markov property, law of large numbers for Markov chains
2. Dynamic programming and optimal control
- Basic principles
- Linear systems and quadratic cost, Ricatti equation
- Utility functions, dynamic portfolio allocation
- Optimal stopping
- Correlated disturbances, state augmentation
Keywords
Markov chains, Markov decision processes, dynamic programming, optimal control
Learning Prerequisites
Required courses
A course in basic probability theory
Important concepts to start the course
Students should be familiar with basic concepts of probability theory, calculus and linear algebra.
Learning Outcomes
By the end of the course, the student must be able to:- Formulate Markov chain models for dynamic uncertain phenomena.
- Formulate Markov decision process models for dynamic decision problems under uncertainty.
- Use these models to structure real decision-making situations.
- Compute relevant performance measures for Markov models.
- Develop an awareness of the manifold uses of probability theory in management science.
Transversal skills
- Communicate effectively, being understood, including across different languages and cultures.
- Assess one's own level of skill acquisition, and plan their on-going learning goals.
Teaching methods
Classical formal teaching interlaced with practical exercices.
Expected student activities
Active participation in exercise sessions is essential.
Assessment methods
- 20% homework
- 30% midterm project
- 50% final exam
Supervision
Office hours | Yes |
Assistants | Yes |
Forum | No |
Resources
Bibliography
Introduction to Probability Models, 10th edition, Sheldon M. Ross, Academic Press, 2009.
Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas, Athena Scientific, 2005.
Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis, Athena Scientific, 2002.
Ressources en bibliothèque
- Dynamic Programming and Optimal Control / Bertsekas
- Introduction to Probability / Bertsekas
- Introduction to Probability Models / Ross
Prerequisite for
Advanced MTE courses
In the programs
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
- SemesterFall
- Exam formWritten
- Credits
4 - Subject examined
Applied probability & stochastic processes - Lecture
2 Hour(s) per week x 14 weeks - Exercises
2 Hour(s) per week x 14 weeks
- Semester
Reference week
Mo | Tu | We | Th | Fr | |
---|---|---|---|---|---|
8-9 | |||||
9-10 | |||||
10-11 | CM2 | ||||
11-12 | |||||
12-13 | |||||
13-14 | |||||
14-15 | CM1 | ||||
15-16 | |||||
16-17 | |||||
17-18 | |||||
18-19 | |||||
19-20 | |||||
20-21 | |||||
21-22 |
legend
- Autumn semester
- Winter sessions
- Spring semester
- Summer sessions
- Lecture in French
- Lecture in English
- Lecture in German