Markov chains

MATH-332 / 5 crédits

Enseignant: Mountford Thomas

Langue: Anglais

Summary

The course follows the text of Norris and the polycopie (which will be distributed chapter by chapter).

Content

We will follow the book of Norris beginning with a recap of basic probability.  Then we pass to the definition of Markov chains and the definition of irreducible.  We analyze notions of recurrence and transcience, particularly for irreducible chains.  We then define positive recurrence and stationary distributions before proving the convergence theorem for aperiodic positive recurrent markov chains.  The last two topics are continuous times Markov Chains and renewal theorms.

Keywords

Stationary distributions.  Irreducibility. Aperiodicity. Communicating classes.  Transcience and recurrance.  Transition matrices.  Operators.

Learning Prerequisites

Required courses

Second year probability.

Learning Outcomes

By the end of the course, the student must be able to:

  • Compute stationary distributions
  • Classify communicating classes
  • Solve hitting probabilities
  • Use the renewal theorem
  • Check irreducibility

Transversal skills

  • Demonstrate the capacity for critical thinking

Teaching methods

Lectures followed by exercise sessions

Assessment methods

The greater part of the note will be determined by the final (written) exam.  There will also be small contribution by a "midterm" exam and by exercises.

Dans le cas de l'art. 3 al. 5 du Règlement de section, l'enseignant décide de la forme de l'examen qu'il communique aux étudiants concernés.

Supervision

Office hours No
Assistants No

Resources

Bibliography

Markov Chains by J. Norris is recommended but not obligatory.

Ressources en bibliothèque

Notes/Handbook

Notes will be made available

Moodle Link

Dans les plans d'études

  • Semestre: Printemps
  • Forme de l'examen: Ecrit (session d'été)
  • Matière examinée: Markov chains
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel
  • Semestre: Printemps
  • Forme de l'examen: Ecrit (session d'été)
  • Matière examinée: Markov chains
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel
  • Semestre: Printemps
  • Forme de l'examen: Ecrit (session d'été)
  • Matière examinée: Markov chains
  • Cours: 2 Heure(s) hebdo x 14 semaines
  • Exercices: 2 Heure(s) hebdo x 14 semaines
  • Type: optionnel

Semaine de référence

Cours connexes

Résultats de graphsearch.epfl.ch.