IFT 6269 : Probabilistic Graphical Models - Fall 2020

Last year version: Fall 2019
Newer version: Fall 2021

Description

This course provides a unifying introduction to statistical modeling of multidimensional data through the framework of probabilistic graphical models, together with their associated learning and inference algorithms.

People

Class info (lectures online)

The lectures will be (synchronous) online on Zoom, but also recorded for further review or for those in remote time zones.

Tentative content

(Detailed outline below)

Evaluation

Prerequisites

Textbook

Homework logistics

Detailed outline (updated often)

Below is a draft detailed outline that will be updated as the class goes on. For now it is the outline recopied from the Fall 2019 version of this class, with the links to the relevant old scribbled notes that will be updated with the new ones gradually. The related chapters in Mike's book are given (but note that they do not exactly correspond with the class content), and also sometimes pointers to the Koller and Friedman book (KF), the Deep Learning book (DL) or the Bishop's book (B). Related past ‘‘scribe notes’’ from the class that I taught in Paris are given for now, and will be updated with the scribe notes as I get them (if I get some).

Date Topics Related chapters
Scribbled notes
Recordings
Scribe notes Homework milestones
Sept 1 (Tu) Set-up & overview intro slides
lecture1
recording
Isabela Albuquerque (Fa17)
lecture1.pdf
source
Sept 4 (Fri) Probability review 2.1.1
DL: 3 (nice and gentle)
KF: 2.1 (more rigorous)
lecture2
recording
William Léchelle (Fa16)
lecture2.pdf
source
Sept 8 (Tu) Prob. review (cont.)
Parametric models
5
lecture3
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture3.pdf
source
Hwk 1 out
(hwk 1 source)
Sept 11 (Fri) Frequentist vs. Bayesian
Maximum likelihood
lecture4
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture4.pdf
source
Sept 15 (Tu) MLE (cont.)
Basic Optimization
Bias-Variance Decomposition
1.3 in Bickel & Doksum
Bias-variance tradeoff: 7.3 in Hastie's book
lecture5
recording
Sébastien Lachapelle (Fa17)
lecture5.pdf
source
Sept 18 (Fri) Statistical decision theory lecture6
recording
same as lecture5
Sept 22 (Tu) Properties of MLE
Linear regression
DL: 7.1 (l2, l1-reg.)
6, 7
lecture7
recording
Zakaria Soliman (Fa16)
lecture6.pdf
source
Hwk 1 due
Hwk 2 out
Sept 25 (Fri) Ridge regression
Logistic regression
7
lecture8
recording
Eeshan Gunesh Dhekane and Younes Driouiche (Fa18)
lecture8.pdf
source
Sept 29 (Tu) Numerical Optimization
Logistic regression (cted) + IRLS

7
DL: 4.3
Boyd's book
lecture9
recording
same as lecture8
Oct 2 (Fri) Gen. classification (Fisher)
Derivative tricks for
Gaussian MLE

Kernel trick (skipped)
7
Matrix Diff. book
lecture10

recording

lecture9 2017 (kernel trick) (skipped)
Eeshan Gunesh Dhekane (Fa18)
lecture9.pdf
source
Oct 6 (Tu) latent variable model
K-means
10
lecture11
recording
same as lecture9
Oct 9 (Fri)
GMM and EM
10,11
lecture12
recording part I
recording part II
Ismael Martinez and Binulal Narayanan (Fa20)
lecture12-2020.pdf
source
Oct 13 (Tu) Graph theory
Directed Graphical Models
2
lecture13
recording
Martin Weiss and Eeshan Gunesh Dhekane (Fa18)
lecture11.pdf
source
Hwk 2 due
Hwk 3 out
(hwk 3 source)
data.zip
Oct 16 (Fri) DGM (cont.)
Undirected graphical models
2
lecture14
recording
Philippe Beardsell (Fa18)
lecture12.pdf
source
Oct 20 (Tu) Break: look at projects
Oct 23 (Fri) Break: look at projects
Oct 27 (Tu) UGM (cont.) 2
lecture15
recording
same as lecture12
Oct 30 (Fri) Inference: elimination alg.
Sum-product alg.
3, 4
lecture16
recording
lecture13-2018.pdf
source

lecture16-2020.pdf
source
Nov 3 (Tu) Max-product
junction tree
HMM
17, 12
lecture17
recording
lecture16-2020.pdf (above)
lecture15-2018.pdf
source
Hwk 3 due
Hwk 4 out
(hwk 4 source)
Nov 6 (Fri) EM for HMM
Information theory
19
lecture18 (corrected)
recording
lecture15-2018.pdf (above)
lecture16-2018.pdf
source
Nov 10 (Tu) Max entropy
MaxENT duality
8
(KL geometry: lecture 16 2017)
lecture19
recording
lecture16-2018.pdf (above) Project: team formed
Nov 13 (Fri) Exponential families
Estimation in graphical models
8
9
lecture20
recording
MVA lecture5
MVA lecture8
Nov 17 (Tu) MC integration
Sampling
21
(variance reduction: see old lecture18 2017)
lecture21
recording
Nov 20 (Fri) MCMC
Markov chains
Metropolis-Hastings
21
lecture22
recording
Nov 24 (Tu) Gibbs sampling
Variational methods
Bishop: 10.1
lecture23
recording

(skipped parts of: old lecture22 2017 for marginal polytope)
MVA lecture9 Hwk 4 due
Hwk 5 out
(hwk 5 source)
Nov 27 (Fri) Bayesian methods
Model selection

Causality
5, 26
lecture24
recording

Causality for Machine Learning (arXiv 2019)
Elements of Causal Inference (book)
MVA lecture10
Dec 1 (Tu) Gaussian networks

Factor analysis, PCA, CCA
(Kalman filter)
VAE
lecture25
recording

13
old lecture17 Fa2016
14 , (15)
old lecture18 Fa2016
VAE – DL: 20.10.3
MVA lecture6.5
MVA lecture7.3
Project: 1 page
progress report due
Dec 4 (Fri) (Jose's lecture) Non-parametric models: Gaussian processes
Dirichlet processes
25
lecture26
recording

GP book
DP tutorial
Dec 8 (Tu) No lecture this week (NeurIPS)
work on your project!
Dec 15 (Tu) Poster presentation
2:00pm-5:00pm
online on Gather.town
Take-home final out (not graded)
Dec 23 (Wed) Hwk 5 due
Project report due