Coursebooks 2017-2018

PDF
 

Caution, these contents corresponds to the coursebooks of last year


Unsupervised and reinforcement learning in neural networks

CS-434

Lecturer(s) :

Gewaltig Marc-Oliver

Language:

English

Summary

Learning is observable in animal and human behavior, but learning is also a topic of computer science. This course links algorithms from machine learning with biological phenomena of synaptic plasticity. The course covers unsupervised and reinforcement learning, but not supervised learning.

Content

I. unsupervised learning
1. Neurons and Synapses in the Brain. Synaptic Changes
2. Biology of unsupervised learning, Hebb rule and LTP .
3. Hebb rule in a linear neuron model and PCA
4. Analysis of Hebb rule and application to development
5. Plasticity and Independent Component Analysis (ICA)
6. Competitive Learning and Clustering
7. Kohonen networks

 

II. Reinforcement learning
8. The paradigm of reward-based learning
in biology and theoretical formalisation
9. Reinforcement learning in discrete spaces
10. Eligibity traces and reinforcement learning in continuous spaces and applications

 

III. Can the brain implement Unsupervised and Reinforcement learning?
11. Spiking neurons and learning: STDP
12. Neuromodulators and Learning
13. Long-term stability of synaptic memory
14. Unsupervised learning from an optimality
viewpoint: Information Maximization

Keywords

synaptic plasticity

learning rules

learning algorithms

neural networks

Learning Prerequisites

Required courses

Analysis I-III, linear algebra, probability and statistics

Recommended courses

Analysis I-III, linear algebra, probability and statistics

Important concepts to start the course

The student needs to be able to use mathematical abstrations as well as linear algebra, probability theory and statistics, analysis and calculus.

Learning Outcomes

By the end of the course, the student must be able to:

Transversal skills

Teaching methods

Classroom teaching, exercises and miniproject

Expected student activities

participate in class (slides are not self-contained)

solve paper and pencil exercises

write and run simulations for miniproject

write report

 

Assessment methods

The final grade is composed of two mini-projects and one exam.

The two mini-projects together count 1/3 of the final grade.

The final exam counts 2/3 of the final grade.

The exam will be written if the course has more than 40 students and oral otherwise.

Supervision

Office hours No
Assistants Yes
Forum Yes

Resources

Bibliography

Dayan & Abbott : Theoretical Neuroscience, MIT Press 2001;

Gerstner & Kistler : Spiking Neuron Models, Cambridge Univ. Press

Sutton & Barto: Reinforcement learning, MIT Press1998

Ressources en bibliothèque
Websites
Moodle Link

In the programs

Reference week

 MoTuWeThFr
8-9     
9-10     
10-11     
11-12     
12-13     
13-14     
14-15     
15-16     
16-17     
17-18     
18-19     
19-20     
20-21     
21-22     
Under construction
 
      Lecture
      Exercise, TP
      Project, other

legend

  • Autumn semester
  • Winter sessions
  • Spring semester
  • Summer sessions
  • Lecture in French
  • Lecture in English
  • Lecture in German