# Fundamentals of inference and learning

## Summary

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and practical computational aspects in python.

## Content

The topics will be chosen from the following basic outline:

- Statistical inference: Estimators, Bias-Variance, Consistency, Efficiency, Maximum likelihood, Fisher Information.
- Bayesian inference, Priors, A posteriori estimation, Expectation-Minimization.
- Supervised learning : Linear Regression, Ridge, Lasso, Sparse problems, high-dimensional Data, Kernel methods, Boosting, Bagging. K-NN, Support Vector Machines, logistic regression, Optimal Margin Classifier
- Statistical learning theory: VC Bounds and Uniform convergence, Implicit regularisation, Double-descent
- Unsupervised learning : Mixture Models, PCA & Kernel PCA, k-means
- Deep learning: multi-layer nets, convnets, auto-encoder, Gradient-descent algorithms
- Basics of Generative models & Reinforcement learning

## Keywords

Statisitics, Supervised and unsupervised learning

## Learning Prerequisites

## Required courses

* Basic probability theory.

* Basic knowlegde of python programing

* Basic linear algebra, and calculus

## Recommended courses

Probability and statistics.

Advance Python

Basic optimization

## Important concepts to start the course

Students should be familiar with basic concepts of probability theory, calculus and linear algebra, and be familiar with python.

## Learning Outcomes

By the end of the course, the student must be able to:

- Formulate statistical models and apply them to statistical learning
- Apply machine learning technics to data science problems
- Solve concrete data science problems
- Explain and understand the fundamental principle of learning theory

## Assessment methods

* Homeworks during the lectures

* Final project,

## In the programs

**Semester:**Fall**Exam form:**During the semester (winter session)**Subject examined:**Fundamentals of inference and learning**Lecture:**2 Hour(s) per week x 14 weeks**Project:**2 Hour(s) per week x 14 weeks**Type:**mandatory

**Semester:**Fall**Exam form:**During the semester (winter session)**Subject examined:**Fundamentals of inference and learning**Lecture:**2 Hour(s) per week x 14 weeks**Project:**2 Hour(s) per week x 14 weeks**Type:**mandatory