Visual intelligence

Summary

This course covers both classic concepts and recent advances in computer vision and machine learning — with a primary focus on embodied intelligence and multimodal learning.

Content

Keywords

Computer vision, Machine learning, Multimodal learning, Embodied intelligence, Robotics, AI.

Learning Prerequisites

Required courses

• CS-233 Introduction to Machine Learning or CS-433 Machine Learning or equivalent course on the basics of machine learning
• CS-456 Deep reinforcement learning or EE-559 Deep Learning or equivalent course on the basics of deep learning

Recommended courses

• CS-442 Computer vision or equivalent undergraduate/master course in the basics of computer

Important concepts to start the course

• Deep learning and machine learning.
• Python programming.
• Basics of probability and statistics.
• Familiarity with RL, for the students who pick projects that involve RL.

Learning Outcomes

By the end of the course, the student must be able to:

• Define basic concepts in computer vision, such as detection, segmentation, 3D from X, as covered in the lectures
• Explain the range of theories in psychology around visual perception, covered in the lectures
• Design and implement computer vision/multimodal learning/machine learning algorithms and foundation models to address problems with real-world complexity
• Design and implement proper evaluation pipelines for computer vision/multimodal learning/machine learning algorithms to assess their performance in the real-world
• Assess / Evaluate the limits and performance pitfalls of a given computer vision/multimodal learning/machine learning algorithm, especially when facing real-world complexity

Transversal skills

• Write a scientific or technical report.
• Make an oral presentation.
• Assess progress against the plan, and adapt the plan as appropriate.
• Demonstrate the capacity for critical thinking

Teaching methods

• Lectures
• Programming notebooks
• Lab sessions
• Project Tutoring
• Course Project

Expected student activities

• In regard to the lectured material, the students are expected to study the provided reading material, actively participate in the class, engage in the discussions, and answer homework questions.
• For the programming homework, students are expected to complete the provided Python notebook assignments.
• In regard to the course project, the students are expected to formulate and implement an in-depth project, demonstrate continuous progress throughout the semester, and provide a final written report and presentation.

Assessment methods

• Project (60%) [distributed over the project proposal, milestone reports, final report and presentation]
• Homeworks (40%)

Supervision

Resources

Bibliography

• Vision Science: Photons to Phenomenology, Steven Palmer, 1999.
• Foundations of Computer Vision, Torralba, Isola, and Freeman, 2024.
• The Ecological Approach to Visual Perception, Jame Gibson, 1979.
• Animal Eyes, Michael Land and Dan-Eric Nilsson, 2012.

Ressources en bibliothèque

• Find the references at the Library

Notes/Handbook

The reference reading of different lectures will be from different books (the main ones listed above) and occasionally from papers. Resources will be provided in class. Full-text books are not mandatory.

Moodle Link

• https://go.epfl.ch/CS-503