Fiches de cours 2017-2018

PDF
 

Applied probability & stochastic processes

MGT-484

Enseignant(s) :

Rujeerapaiboon Napat

Langue:

English

Summary

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc.

Content

The following topics will tentatively be covered in the course:

 

1. Discrete-time Markov chains

- Basic definitions, transition probabilities

- Classification of states

- Stationary and limiting distributions, convergence to equilibrium

- Hitting times and absorption probabilities

- Strong Markov property, law of large numbers for Markov chains

 

2. Dynamic programming and optimal control

- Basic principles

- Linear systems and quadratic cost, Ricatti equation

- Utility functions, dynamic portfolio allocation

- Optimal stopping

- Correlated disturbances, state augmentation

Keywords

Markov chains, Markov decision processes, dynamic programming, optimal control

Learning Prerequisites

Required courses

A course in basic probability theory

Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra.

Learning Outcomes

By the end of the course, the student must be able to:

Transversal skills

Teaching methods

Classical formal teaching interlaced with practical exercices.

Expected student activities

Active participation in exercise sessions is essential.

Assessment methods

30% midterm exam

70% final exam

Supervision

Office hours Yes
Assistants Yes
Forum No

Resources

Bibliography

Introduction to Probability Models, 10th edition, Sheldon M. Ross, Academic Press, 2009.

Dynamic Programming and Optimal Control, 3rd edition, Dimitri P. Bertsekas, Athena Scientific, 2005.

Introduction to Probability, Dimitri P. Bertsekas and John N. Tsitsiklis, Athena Scientific, 2002.

 

Ressources en bibliothèque

Prerequisite for

Advanced MTE courses

Dans les plans d'études

Semaine de référence

 LuMaMeJeVe
8-9     
9-10     
10-11  GRB330  
11-12    
12-13     
13-14     
14-15 ODY016   
15-16    
16-17     
17-18     
18-19     
19-20     
20-21     
21-22     
 
      Cours
      Exercice, TP
      Projet, autre

légende

  • Semestre d'automne
  • Session d'hiver
  • Semestre de printemps
  • Session d'été
  • Cours en français
  • Cours en anglais
  • Cours en allemand