IFT 6269 : Probabilistic Graphical Models - Fall 2019

For the 2020 edition of the class - click here.

Description

This course provides a unifying introduction to statistical modeling of multidimensional data through the framework of probabilistic graphical models, together with their associated learning and inference algorithms.

People

Class info

Tentative content

(Detailed outline below)

Evaluation

Prerequisites

Textbook

Homework logistics

Detailed outline (updated often)

Below is a draft detailed outline that will be updated as the class goes on. For now it is the outline recopied from the Fall 2018 version of this class, with the links to the relevant old scribbled notes that will be updated with the new ones gradually. The related chapters in Mike's book are given (but note that they do not exactly correspond with the class content), and also sometimes pointers to the Koller and Friedman book (KF), the Deep Learning book (DL) or the Bishop's book (B). Related past ‘‘scribe notes’’ from the class that I taught in Paris are given for now, and will be updated with the scribe notes as I get them (if I get some).

Date Topics Related chapters
Scribbled notes
Scribe notes Homework milestones
Sept 3 Set-up & overview intro slides
lecture1
Isabela Albuquerque (Fa17)
lecture1.pdf
source
Sept 6 Probability review 2.1.1
DL: 3 (nice and gentle)
KF: 2.1 (more rigorous)
lecture2
William Léchelle (Fa16)
lecture2.pdf
source
Sept 10 Parametric models
Frequentist vs. Bayesian
5
lecture3
Philippe Brouillard and Tristan Deleu (Fa17)
lecture3.pdf
source
Hwk 1 out
(hwk 1 source)
Sept 13 Bayesian (cont.)
Maximum likelihood
lecture4 Philippe Brouillard and Tristan Deleu (Fa17)
lecture4.pdf
Sept 17 MLE (cont.)
Statistical decision theory
1.3 in Bickel & Doksum
Bias-variance tradeoff: 7.3 in Hastie's book
lecture5
Sébastien Lachapelle (Fa17)
lecture5.pdf
source
Sept 20 Properties of estimators lecture6 same as lecture5
Sept 24 Linear regression
Logistic regression
DL: 7.1 (l2, l1-reg.)
6, 7
lecture7
Zakaria Soliman (Fa16)
lecture6.pdf
source
Hwk 1 due
Hwk 2 out
data.zip
Sept 27 Class cancelled: climate march
Oct 1 Optimization
Logistic regression (cted) + IRLS
7
DL: 4.3
Boyd's book
lecture8
MVA lecture2
Oct 4 Gen. classification (Fisher)
Derivative tricks for
Gaussian MLE
Kernel trick (skipped)
Matrix Diff. book
10, 11
lecture9 2017 (kernel trick)
lecture9
MVA lecture3
Oct 8 K-means
GMM and EM
10,11
lecture10
Oct 11 Graph theory
Directed graphical models
2
lecture11
MVA lecture4
Oct 15 DGM (cont.) 2
lecture12
Hwk 2 due
Hwk 3 out
(hwk 3 source)
data.zip
Oct 18 Undirected graphical models
Inference: elimination alg.
3
lecture13
Oct 22 Sum-product alg. 4
lecture14
MVA lecture7
Oct 25 Break: look at projects
Oct 29 Max-product
junction tree
HMM
17, 12
lecture15
Nov 1 Class cancelled: look at projects
Nov 5 Class cancelled: look at projects Hwk 3 due
Hwk 4 out
(hwk 4 source)
Nov 8 EM for HMM
Information theory
Max entropy
19
lecture16
MVA lecture5
Nov 12 MaxENT duality
Exponential families
8
(KL geometry: lecture 16 2017)
lecture17
MVA lecture6 Project: team formed
Nov 15 Exponential families (cont.)
Estimation in graphical models
MC integration
8
9
21
lecture18
MVA lecture8
Nov 19 Sampling 21
(variance reduction: see old lecture18 2017)
lecture19
Nov 22 MCMC
Markov chains
Metropolis-Hastings
21
lecture20
Nov 26 Gibbs sampling
Variational methods
Bishop: 10.1
lecture21

(skipped parts of: old lecture22 2017)
MVA lecture9 Hwk 4 due
Hwk 5 out
(hwk 5 source)
Nov 29 Bayesian methods
Model selection

Causality
5, 26
lecture22

Causality for Machine Learning (arXiv 2019)
Elements of Causal Inference (book)
MVA lecture10
Dec 3 Gaussian networks

Factor analysis, PCA, CCA
(Kalman filter)
VAE
lecture23

13
old lecture17 Fa2016
14 , (15)
old lecture18 Fa2016
VAE – DL: 20.10.3
MVA lecture6.5
MVA lecture7.3
Project: 1 page
progress report due
Dec 6 (Jose's lecture) Non-parametric models: Gaussian processes
Dirichlet processes
25
lecture24
GP book
DP tutorial
Dec 10 No lecture this week (NeurIPS)
work on your project!
Dec 17 Poster presentation
Time and place TBC (most likely 1:00pm-4:00pm)
Hwk 5 due
Take-home final out
Dec 23 Project report due
Take-home final due (online)