Syllabus and Course Schedule
Some of the following slides will be updated soon.
Week 
Lecture 
Topics 
References 
Announcements 
Week 1 
Introduction to ML [slides] 
What is machine learning? What are the basic elements of machine learning? Introduction to decision trees and instancebased learning. 
Book: Pattern Recognition and Machine Learning, Bishop [link] Book: Probabilistic Reasoning in Intelligent Systems, Judea Pearl [link] Book: Probabilistic Graphical Models, D. Koller et al [link] 

Week 2 
MLE, Bayesian and MAP Estimation [slides] 
What is MLE? What is Bayesian inference? How to choose the priors? What is MAP estimation?



Week 3 
Generative and Discriminative Learning [slides] 
What are generative models? What are discriminative models? Naïve Bayes and logistic regression. Theoretical analysis of generative and discriminative models. 

Hw1 is out. 
Week 4 
Learning Theory [slides] 
BiasVariance Tradeoff Model complexity and its influence on generalization performance VC theory 


Week 5 
Support vector machines I [slides] 
Geometric view of margin and the basic principle Hard margin and soft margin support vector machines Multiclass support vector machines Basics of constrained optimization


Hw2 is out. 
Week 6 
Support Vector Machines II [slides] 
The kernel trick Structured Support Vector Machines 
Book: The nature of statistical learning theory, V. N. Vapnik [link] A Tutorial on Support Vector Machines for Pattern Recognition, Christopher J.C. Burges [link] An Introduction to Support Vector Machines, S. John et al. [link] 

Week 7 
Online Learning and the Perceptron Algorithm [slides] 
Online learning algorithms The perceptron algorithm 


Week 8 
Clustering and EM [slides] 
Kmeans Gaussian Mixture Models EM algorithms

A Gentle Tutorial on the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, J. A. Bilmes [link] 

Week 9 
HMM and Graphical Models I [slides] 
HMM and general graphical model representations Exact inference algorithms

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, J. Lafferty et al. [link] A maximum entropy approach to natural language processing, A. L. Berger et al. [link] Maximum Entropy Markov Models for Information Extraction and Segmentation, A. McCallum et al. [link] 
Hw3 is out. 
Week 10 
Graphical Models II [slides] 
Parameter learning and structure learning Inference with latent variables Variational Methods 
Book: Graphical Models, Exponential Families, and Variational Inference, M. J. Wainwright et al. [link] 

Week 11 
Basics of Nonparametric Bayesian Methods and DP Mixtures [slides] 
Basics of nonparametric methods and nonparametric Bayesian methods Finite Bayesian Gaussian mixture models Dirichlet process mixture models

Markov chain sampling methods for Dirichlet process mixture models, R. M. Neal [link] Variational inference for Dirichlet process mixtures, D. M. Blei et al. [link] Hierarchical Dirichlet processes Y. W. Teh et al. [link] 
Hw4 is out. 
Week 12 
Monte Carlo Methods [slides] 
Basic Monte Carlo sampling methods Markov chain Monte Carlo methods

Chap. 11 of Pattern Recognition and Machine Learning, Bishop [link] Introduction to Monte Carlo Methods. D.J.C. [link] 

Week 13 
Topic Models [slides] 
Latent semantic analysis and probabilistic latent semantic analysis Hierarchical Bayesian topic models 
Latent Dirichlet Allocation, JMLR, D. M. Blei et al. [link] Finding Scientific Topics, T. L. Griffiths et al. [link] Estimating a Dirichlet Distribution, T. P. Minka [link] 

Week 14 
Feature Selection and Sparse Learning in High Dimensions [slides] 
Basic methods for feature selection Sparse learning with L1 norms Learning with structured sparsity 
Regression Shrinkage and Selection via the Lasso, R. Tibshiranit [link] Optimization with SparsityInducing Penalties, F. Bach et al. [link] 

Week 15 
Advanced topics and Summary [slides] 

Emergence of simplecell receptive field properties by learning a sparse code for natural images, B. A. Olshausen [link] Online Learning for Matrix Factorization and Sparse Coding, J. Mairal et al. [link] 

Week 16 
Project Presentations 


