# Coursebooks 2016-2017

## Pattern classification and machine learning

#### Lecturer(s) :

Jaggi Martin Lukas
Urbanke Rüdiger

English

#### Summary

Pattern classification occupies a central role in machine learning from data. In this course, basic principles and methods underlying machine learning will be introduced. The student will learn few basic methods and their relations to each other.

#### Content

1. Basic regression and classification methods: Linear regression, Ridge regression, logistic regression, and k-NN.
2. Basic concepts: cost-functions and optimization, corss-validation and bias-variance trade-off, curse of dimensionality.
3. Advanced regression and classification methods: generalized linear model, SVM and Kernel methods, Gaussian processes and Bayesian methods, Neural network and deep learning, random forest and boosting.
4. Clustering: Mixture model, k-means, Gaussian mixture model and EM algorithm.
5. Dimensionality reduction: PCA and matrix factorization.
6. Time-series: Bayesian network, Kalman filters and HMM, belief propagation.

#### Learning Prerequisites

##### Required courses

• Analysis I, II, III
• Linear Algebra
• Probability and Statistics (MATH-232)
• Algorithms (CS-250)

##### Recommended courses

• Introduction to differentiable optimization (MATH-265)
• Linear Models (MATH-341)

##### Important concepts to start the course

• Programming in Matlab (basic skills)
• Basic probability and statistics (conditional and joint distribution, independence, Bayes rule, random variables, expectation, mean, median, mode, central limit theorem)
• Basic linear algebra (system of linear equations and SVD)
• Basic multivariate calculus (derivative wrt vector and matrix)
• Univariate and multivariate Gaussian distribution (joint, conditional, and marginals)

#### Learning Outcomes

By the end of the course, the student must be able to:
• Define the following basic machine learning problems: Regression, classification, clustering, dimensionality reduction, time-series
• Explain main differences between them.
• Describe a few important models and algorithms for them.
• Implement these methods
• Apply them to real-world problems
• Compare their performaces
• Design new methods
• Choose for the real-world problem in hand
• Critique them
• Defend them
• Derive the theory behind ML methods taught in the course
• Generalize them to new problems

#### Transversal skills

• Continue to work through difficulties or initial failure to find optimal solutions.
• Assess one's own level of skill acquisition, and plan their on-going learning goals.

• Lectures
• Lab sessions
• Projects

#### Expected student activities

• Students MUST attend lectures every week and take notes during the lecture
• Students MUST attend lab sessions every week and write their own code
• Students MUST work on projects where they use the code developed during labs
• Students should read lecture note and complete weekly assignments

#### Assessment methods

• Continuous control
• Final exam

#### Resources

##### Bibliography

The following books will be used for further readings.

##### Notes/Handbook

The course comes with partially-filled lecture notes which will be available to students before each lecture. These notes will NOT be complete and students are supposed to complete them during/after a lecture. This way students will be able to create their own written notes on top of the one provided to them.

### Reference week

MoTuWeThFr
8-9
9-10
10-11
11-12
12-13
13-14
14-15
15-16
16-17
17-18
18-19
19-20
20-21
21-22
Under construction

Lecture
Exercise, TP
Project, other

### legend

• Autumn semester
• Winter sessions
• Spring semester
• Summer sessions
• Lecture in French
• Lecture in English
• Lecture in German