Markov chains
Summary
The course follows the text of Norris and the polycopie (which will be distributed chapter by chapter).
Content
We will follow the book of Norris beginning with a recap of basic probability. Then we pass to the definition of Markov chains and the definition of irreducible. We analyze notions of recurrence and transcience, particularly for irreducible chains. We then define positive recurrence and stationary distributions before proving the convergence theorem for aperiodic positive recurrent markov chains. The last two topics are continuous times Markov Chains and renewal theorms.
Keywords
Stationary distributions. Irreducibility. Aperiodicity. Communicating classes. Transcience and recurrance. Transition matrices. Operators.
Learning Prerequisites
Required courses
Second year probability.
Learning Outcomes
By the end of the course, the student must be able to:
- Compute stationary distributions
- Classify communicating classes
- Solve hitting probabilities
- Use the renewal theorem
- Check irreducibility
Transversal skills
- Demonstrate the capacity for critical thinking
Teaching methods
Lectures followed by exercise sessions
Assessment methods
The greater part of the note will be determined by the final (written) exam. There will also be small contribution by a "midterm" exam and by exercises.
Dans le cas de l'art. 3 al. 5 du Règlement de section, l'enseignant décide de la forme de l'examen qu'il communique aux étudiants concernés.
Supervision
| Office hours | No |
| Assistants | No |
Resources
Bibliography
Markov Chains by J. Norris is recommended but not obligatory.
Ressources en bibliothèque
Notes/Handbook
Notes will be made available
Moodle Link
In the programs
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Markov chains
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Markov chains
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Markov chains
- Courses: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Type: optional