# Statistical inference and machine learning

MGT-448 / **4 credits**

**Teacher: **

**Language:** English

## Summary

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data mining.

## Content

List of topics:

- General Introduction
- Supervised Learning, Discriminative Algorithms:

Supervised Learning Concept, Linear Regression, Maximum Likelihood, Normal Equation Gradient Descent, Stochastic Gradient, SVRG.

Linear Classification, Logistic Regression, Newton Method, - Generative Algorithms:

Multivariate Normal, Linear Discriminant Analysis

Naive Bayes, Laplacian Smoothing

Multiclass Classification, K-NN

Multi-class Fisher Discriminant Analysis, Multinomial Regression

Support Vector Machines and Kernel Methods:

Intuition, Geometric Margins, Optimal Margin Classifier

Lagrangian Duality, Soft-margin, Loss function, Stochastic Subgradient Method. Kernel, SMO algorithm, Coordinate Gradient Descent.

Kernel PCA, Kernel Logistic Regression, Kernel Ridge Regression, Multiclass SVM - Unsupervised Learning:

PCA, Mixture Models, Bayesian Graphical Models

Power Method, Ojaâ€™s algorithm, EM Algorithm, Variational Inference Matrix Factorization/Completion - Regularization and Model Selection:

Cross Validation, Hill Climbing, Bayesian Optimization Bayesian Regression, Bayesian Logistic Regression

Forward and Backward Regression, Lasso, elastic-net. Proximal Gradient, Prox-SVRG.

Coordinate Proximal Gradient, Pathwise Coordinate Descent - Decision Tree and Random Forest:

Entropy, Building Tree

Bagging features, Bagging Samples, Random Forest Adaboost, Gradient Tree Boosting - Neural Network:

Concept; Deep Neural Network; Backpropagation Convolutional Neural Network;

## Keywords

Supervised and unsupervised learning, Model selection, Generative models.

## Learning Prerequisites

## Required courses

A course in basic probability theory.

## Recommended courses

linear algebra and statistics.

## Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra.

## Learning Outcomes

By the end of the course, the student must be able to:

- Formalize Formulate supervised and unsupervised learning problems and apply it to data.
- Understand and apply generative models.
- Understand and train basic neural networks and apply them to data.

## Transversal skills

- Assess one's own level of skill acquisition, and plan their on-going learning goals.

## Teaching methods

Classical formal teaching interlaced with practical exercices.

## Expected student activities

Active participation in exercise sessions is essential.

## Assessment methods

30% Homework

20% Midterm project

50% Final project

## Supervision

Office hours | Yes |

Assistants | Yes |

Forum | No |

## In the programs

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

**Semester:**Fall**Exam form:**Written (winter session)**Subject examined:**Statistical inference and machine learning**Lecture:**2 Hour(s) per week x 14 weeks**Exercises:**2 Hour(s) per week x 14 weeks

## Reference week

Mo | Tu | We | Th | Fr | |

8-9 | |||||

9-10 | |||||

10-11 | |||||

11-12 | |||||

12-13 | |||||

13-14 | |||||

14-15 | |||||

15-16 | |||||

16-17 | |||||

17-18 | |||||

18-19 | |||||

19-20 | |||||

20-21 | |||||

21-22 |