Learning in neural networks

CS-479 / 6 credits

Teacher: Gerstner Wulfram

Language: English

Summary

Full title: "Brain-style learning in Neural Networks: Learning algorithms of the brain". Biological brains show powerful learning without BackProp, how? By a smart combination of Reinforcement Learning and Self-supervised learning with local learning rules at the connections (synapses).

Content

- Why BackProp is biologically not plausible. Two-factor and three-factor rules  in biology and neuromorphic hardware (Synaptic Plasticity/Biology)
- Three-factor rules for reward-based learning (Reinforcement Learning 1)
- Three-factor rules for TD learning: SARSA and eligibility traces (Reinforcement Learning 2)

- Policy gradient(Reinforcement Learning 3)
- Actor-critic networks (Reinforcement Learning 4)
- Reinforcement learning in the brain (Reinforcement Learning 5)

 

- Hebbian two-factor rules (Self-supervised Learning 1)

- Two-factor rules for independent factors (Self-supervised Learning 2)

- Learning of representations in multi-layer networks (Self-supervised Learning 3)


- Learning to find a goal: a bio-plausible model with place cells and rewards (Applications 1)

- Learning by surprise and novelty: exploration and changing environments (Application 2)
- Surprise and novelty in changing environments  (Application 3)
- Neuromorphic hardware and in-memory computing (Application 4)

Keywords

-  Reinforcement Learning  (RL)

- eligibility traces

- surprise and novelty

- two-factor rules and three-factor rules

- synaptic plasticity

- self-supervised learning

- representation learning

- neuromodulators (dopamine)

- neuromorphic hardware

Learning Prerequisites

Required courses

Linear Algebra AND Analysis.

Machine learning

Important concepts to start the course

Optimization, Gradient Descent, Filtering, Loss function, PCA

Learning Outcomes

By the end of the course, the student must be able to:

  • Translate concepts from machine learning into bio-plausible algorithms
  • Translate neuroscience of learning into algorithms
  • Explain differences between and similarities of various algorithms
  • Discriminate limitations and advantages of various learning algorithms for implementation in biology or hardware

Transversal skills

  • Plan and carry out activities in a way which makes optimal use of available time and other resources.
  • Set objectives and design an action plan to reach those objectives.
  • Evaluate one's own performance in the team, receive and respond appropriately to feedback.
  • Give feedback (critique) in an appropriate fashion.
  • Manage priorities.
  • Continue to work through difficulties or initial failure to find optimal solutions.

Teaching methods

Ex cathedra, Exercises, and Miniproject

Expected student activities

Participation in Class, Solution of Exercises, Miniproject.

Assessment methods

Oral  exam (70 percent) plus miniproject (30 percent). If more than 32 students participate, the oral exam is replaced by a written exam. The oral exam  consists of paper presentation (12 min) followed by questions to the paper and contents of class (12 min).

 

Supervision

Office hours No
Assistants Yes
Forum Yes

Resources

Moodle Link

Videos

In the programs

  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional
  • Semester: Spring
  • Exam form: Oral (summer session)
  • Subject examined: Learning in neural networks
  • Courses: 2 Hour(s) per week x 14 weeks
  • Exercises: 1 Hour(s) per week x 14 weeks
  • Lab: 1 Hour(s) per week x 14 weeks
  • Type: optional

Reference week

Related courses

Results from graphsearch.epfl.ch.