Applied probability & stochastic processes
Summary
This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc.
Content
Keywords
Markov chains, Markov decision processes, dynamic programming, optimal control
Learning Outcomes
By the end of the course, the student must be able to:
- Formulate Markov chain models for dynamic uncertain phenomena.
- Formulate Markov decision process models for dynamic decision problems under uncertainty.
- Use these models to structure real decision-making situations.
- Compute relevant performance measures for Markov models.
- Develop an awareness of the manifold uses of probability theory in management science.
- Reason about the concept of a discrete-time Markov chain and understand how Markov chains are used to model random phenomena
- Compute several properties of a given Markov chain, such as hitting probabilities, expected hitting times, invariant distributions and the long-run proportion of time spent in a given state
- Formalize decision problems under uncertainty as optimal control models
- Solve optimal control models via dynamic programming
- Analyze the technical literature in applied probability and to undertake independent self-study (or research) in the future
Transversal skills
- Communicate effectively, being understood, including across different languages and cultures.
- Assess one's own level of skill acquisition, and plan their on-going learning goals.
- Use a work methodology appropriate to the task.
- Demonstrate the capacity for critical thinking
Teaching methods
Classical formal teaching interlaced with practical exercices.
Expected student activities
Active participation in exercise sessions is essential.
Assessment methods
- 30% midterm exam
- 70% final exam
Resources
Bibliography
Introduction to Probability Models, 10th edition, Sheldon M. Ross, Academic Press, 2009.
Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas, Athena Scientific, 2005.
Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis, Athena Scientific, 2002.
Ressources en bibliothèque
- Introduction to Probability Models, 10th edition, Sheldon M. Ross
- Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas
- Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis
Moodle Link
Prerequisite for
Advanced MTE courses
Dans les plans d'études
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: obligatoire
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: obligatoire
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Ecrit (session d'hiver)
- Matière examinée: Applied probability & stochastic processes
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel