Artificial neural networks/reinforcement learning
Summary
Since 2010 approaches in deep learning have revolutionized fields as diverse as computer vision, machine learning, or artificial intelligence. This course gives a systematic introduction into influential models of deep artificial neural networks, with a focus on Reinforcement Learning.
Content
- General Introduction: Deep Networks and Reinforcement Learning
- Reinforcement Learning 1: Bellman equation and SARSA
- Reinforcement Learning 2: variants of SARSA, Q-learning, n-step-TD learning
- Reinforcement Learning 3: Policy gradient
- Deep Networks 1: BackProp, Multilayer Perceptrons, automatic differentiation
- Deep Networks 2: Tricks of the Trade in deep learning
- Deep Networks 3: Loss landscape and optimization methods for deep networks
- Deep reinforcement learning 1: DeepQ and Actor-Critic, Inductive Bias
- Deep reinforcement learning 2: Eligibility traces from Policy Gradient, Model-free
- Deep reinforcement learning 3: Atari games, Replay buffer, and robotics
- Deep reinforcement learning 4: Model-Based Deep RL
- Deep reinforcement learning 5: Exploration: novelty, surprise, information gain
- Biology and reinforcement learning: Three-factor learning rules
- Hardware, energy consumption, and three-factor learning rules
Keywords
Deep learning, artificial neural networks, reinforcement learning, TD learning, SARSA, Actor-Critic Networks
Learning Prerequisites
Required courses
CS 433 Machine Learning (or equivalent)
Calculus, Linear Algebra (at the level equivalent to first 2 years of EPFL in STI or IC, such as Computer Science, Physics or Electrical Engineering)
Recommended courses
stochastic processes
optimization
Important concepts to start the course
- Regularization in machine learning,
- Training base versus Test base, cross validation.
- Gradient descent. Stochastic gradient descent.
- Expectation, Poisson Process, Bernoulli Process.
Learning Outcomes
By the end of the course, the student must be able to:
- Apply learning in deep networks to real data
- Assess / Evaluate performance of learning algorithms
- Elaborate relations between different mathematical algorithms of learning
- Judge limitations of algorithms
- Propose algoriothms and models for learning from experience
- Apply Reinforcement Learning
Transversal skills
- Continue to work through difficulties or initial failure to find optimal solutions.
- Access and evaluate appropriate sources of information.
- Write a scientific or technical report.
- Manage priorities.
Teaching methods
Ex cathedra lectures and miniproject. Every week the ex cathedra lectures are interrupted for at least one in-class exercise which is then discussed in classroom before the lecture continues. Additional exercises are given as homework or can be disussed in the second exercise hour.
Expected student activities
Work on miniproject
Solve all exercises
Attend all lectures and take notes during lecture, participate in quizzes.
If you cannot attend a lecture, then you must read the recommended book chapters
Assessment methods
Written exam (70 percent) and miniproject (30 percent)
Supervision
Office hours | Yes |
Assistants | Yes |
Forum | Yes |
Others |
Resources
Bibliography
- Textbook: Deep Learning by Goodfellow, Bengio, Courville (MIT Press)
- Textbook: Reinforcement Learning by Sutton and Barto (MIT Press)
Pdfs of the preprint version for both books are availble online
Ressources en bibliothèque
Websites
- http://for videos and lecture slides https://lcnwww.epfl.ch/gerstner/VideoLecturesANN-Gerstner.html
- http://main web page is moodle
Videos
In the programs
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: Written (summer session)
- Subject examined: Artificial neural networks/reinforcement learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks