Fundamentals of inference and learning
Summary
This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and practical computational aspects in python.
Content
The topics will be chosen from the following basic outline:
- Statistical inference: Estimators, Bias-Variance, Consistency, Efficiency, Maximum likelihood, Fisher Information.
- Bayesian inference, Priors, A posteriori estimation, Expectation-Minimization.
- Supervised learning : Linear Regression, Ridge, Lasso, Sparse problems, high-dimensional Data, Kernel methods, Boosting, Bagging. K-NN, Support Vector Machines, logistic regression, Optimal Margin Classifier
- Statistical learning theory: VC Bounds and Uniform convergence, Implicit regularisation, Double-descent
- Unsupervised learning : Mixture Models, PCA & Kernel PCA, k-means
- Deep learning: multi-layer nets, convnets, auto-encoder, Gradient-descent algorithms
- Basics of Generative models & Reinforcement learning
Keywords
Statisitics, Supervised and unsupervised learning
Learning Prerequisites
Required courses
* Basic probability theory.
* Basic knowlegde of python programing
* Basic linear algebra, and calculus
Recommended courses
Probability and statistics.
Advance Python
Basic optimization
Important concepts to start the course
Students should be familiar with basic concepts of probability theory, calculus and linear algebra, and be familiar with python.
Learning Outcomes
By the end of the course, the student must be able to:
- Formulate statistical models and apply them to statistical learning
- Apply machine learning technics to data science problems
- Solve concrete data science problems
- Explain and understand the fundamental principle of learning theory
Assessment methods
* Homeworks during the lectures
* Final project,
In the programs
- Semester: Fall
- Exam form: During the semester (winter session)
- Subject examined: Fundamentals of inference and learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory
- Semester: Fall
- Exam form: During the semester (winter session)
- Subject examined: Fundamentals of inference and learning
- Lecture: 2 Hour(s) per week x 14 weeks
- Project: 2 Hour(s) per week x 14 weeks
- Type: mandatory