Fiches de cours

Statistical inference and machine learning

Kiyavash Negar

English

Summary

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data mining.

Content

List of topics:

• General Introduction
• Supervised Learning, Discriminative Algorithms:
Supervised Learning Concept, Linear Regression, Maximum Likelihood, Normal Equation Gradient Descent, Stochastic Gradient, SVRG.
Linear Classification, Logistic Regression, Newton Method,
• Generative Algorithms:
Multivariate Normal, Linear Discriminant Analysis
Naive Bayes, Laplacian Smoothing
Multiclass Classification, K-NN
Multi-class Fisher Discriminant Analysis, Multinomial Regression
Support Vector Machines and Kernel Methods:
Intuition, Geometric Margins, Optimal Margin Classifier
Lagrangian Duality, Soft-margin, Loss function, Stochastic Subgradient Method. Kernel, SMO algorithm, Coordinate Gradient Descent.
Kernel PCA, Kernel Logistic Regression, Kernel Ridge Regression, Multiclass SVM
• Unsupervised Learning:
PCA, Mixture Models, Bayesian Graphical Models
Power Method, Ojaâs algorithm, EM Algorithm, Variational Inference Matrix Factorization/Completion
• Regularization and Model Selection:
Cross Validation, Hill Climbing, Bayesian Optimization Bayesian Regression, Bayesian Logistic Regression
Forward and Backward Regression, Lasso, elastic-net. Proximal Gradient, Prox-SVRG.
Coordinate Proximal Gradient, Pathwise Coordinate Descent
• Decision Tree and Random Forest:
Entropy, Building Tree
• Neural Network:
Concept; Deep Neural Network; Backpropagation Convolutional Neural Network;

Keywords

Supervised and unsupervised learning, Model selection, Generative models.

Learning Prerequisites

Required courses

A course in basic probability theory.

Recommended courses

linear algebra and statistics.

Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra.

Learning Outcomes

By the end of the course, the student must be able to:
• Formalize Formulate supervised and unsupervised learning problems and apply it to data.
• Understand and apply generative models.
• Understand and train basic neural networks and apply them to data.

Transversal skills

• Assess one's own level of skill acquisition, and plan their on-going learning goals.

Teaching methods

Classical formal teaching interlaced with practical exercices.

Expected student activities

Active participation in exercise sessions is essential.

Assessment methods

30% Homework

20% Midterm project

20% Final project

30% Final exam

Supervision

 Office hours Yes Assistants Yes Forum No

Semaine de référence

LuMaMeJeVe
8-9
9-10
10-11  BS150
11-12
12-13
13-14BS150
14-15
15-16
16-17
17-18
18-19
19-20
20-21
21-22

Cours
Exercice, TP
Projet, autre

légende

• Semestre d'automne
• Session d'hiver
• Semestre de printemps
• Session d'été
• Cours en français
• Cours en anglais
• Cours en allemand