Statistical inference and machine learning

List of topics:

• General Introduction
• Supervised Learning, Discriminative Algorithms:
  Supervised Learning Concept, Linear Regression, Maximum Likelihood, Normal Equation Gradient Descent, Stochastic Gradient, SVRG.
  Linear Classification, Logistic Regression, Newton Method,
• Generative Algorithms:
  Multivariate Normal, Linear Discriminant Analysis
  Naive Bayes, Laplacian Smoothing
  Multiclass Classification, K-NN
  Multi-class Fisher Discriminant Analysis, Multinomial Regression
  Support Vector Machines and Kernel Methods:
  Intuition, Geometric Margins, Optimal Margin Classifier
  Lagrangian Duality, Soft-margin, Loss function, Stochastic Subgradient Method. Kernel, SMO algorithm, Coordinate Gradient Descent.
  Kernel PCA, Kernel Logistic Regression, Kernel Ridge Regression, Multiclass SVM
• Unsupervised Learning:
  PCA, Mixture Models, Bayesian Graphical Models
  Power Method, Oja’s algorithm, EM Algorithm, Variational Inference Matrix Factorization/Completion
• Regularization and Model Selection:
  Cross Validation, Hill Climbing, Bayesian Optimization Bayesian Regression, Bayesian Logistic Regression
  Forward and Backward Regression, Lasso, elastic-net. Proximal Gradient, Prox-SVRG.
  Coordinate Proximal Gradient, Pathwise Coordinate Descent
• Decision Tree and Random Forest:
  Entropy, Building Tree
  Bagging features, Bagging Samples, Random Forest Adaboost, Gradient Tree Boosting
• Neural Network:
  Concept; Deep Neural Network; Backpropagation Convolutional Neural Network;