IFT 6269 : Probabilistic Graphical Models - Fall 2022

Last year version: Fall 2021
Next year version: Fall 2023

Description

This course provides a unifying introduction to statistical modeling of multidimensional data through the framework of probabilistic graphical models, together with their associated learning and inference algorithms.

People

Class info (hybrid lectures)

The lectures are taught in hybrid mode: the lectures will be given at the Mila Agora, 6650 rue Saint-Urbain, but also connected synchronously on Zoom (see info on Studium). They will also be recorded for further review or for those in remote time zones.

Tentative content

(Detailed outline below)

Evaluation

Prerequisites

Textbook

Homework logistics

Detailed outline (updated often)

Below is a draft detailed outline that will be updated as the class goes on. For now it is the outline recopied from the Fall 2021 version of this class, with the links to the relevant old scribbled notes that will be updated with the new ones gradually. The related chapters in Mike's book are given (but note that they do not exactly correspond with the class content), and also sometimes pointers to the Koller and Friedman book (KF), the Deep Learning book (DL) or the Bishop's book (B). Related past ‘‘scribe notes’’ from the previous instantiations of this class as well as (for later lectures) the class that I taught in Paris are given for now, and will be updated with the more updated scribe notes if needed (and I get some).

Date Topics Related chapters
Scribbled notes
Recordings
Scribe notes Homework milestones
Sept 6 (Tu) Set-up & overview lecture1
recording
Isabela Albuquerque (Fa17)
lecture1.pdf
source
Sept 9 (Fri) Probability review 2.1.1
DL: 3 (nice and gentle)
KF: 2.1 (more rigorous)
lecture2
recording
William Léchelle (Fa16)
lecture2.pdf
source
Sept 13 (Tu) Prob. review (cont.)
Parametric models
5
lecture3
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture3.pdf
source
Hwk 1 out
(hwk 1 source)
Sept 16 (Fri) Frequentist vs. Bayesian
Maximum likelihood
lecture4
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture4.pdf
source
Sept 20 (Tu) MLE (cont.)
Statistical decision theory
1.3 in Bickel & Doksum
Bias-variance tradeoff: 7.3 in Hastie's book
lecture5
recording
Sébastien Lachapelle (Fa17)
lecture5.pdf
source
Sept 23 (Fri) Properties of estimators
and MLE
lecture6
recording (2021)
same as lecture5
Sept 27 (Tu) Linear regression
Logistic regression
DL: 7.1 (l2, l1-reg.)
6, 7
lecture7
recording
Zakaria Soliman (Fa16)
lecture6.pdf
source
Hwk 1 due
Hwk 2 out
Sept 30 (Fri) Numerical Optimization
Logistic regression (cted) and IRLS

7
DL: 4.3
Boyd's book
lecture8
recording
Eeshan Gunesh Dhekane and Younes Driouiche (Fa18)
lecture8.pdf
source
Oct 4 (Tu) Gen. classification (Fisher)
Derivative tricks for
Gaussian MLE

Kernel trick (skipped)
7
Matrix Diff. book
lecture9

recording

lecture9 2017 (kernel trick) (skipped)
Eeshan Gunesh Dhekane (Fa18)
lecture9.pdf
source
Oct 7 (Fri) latent variable model
K-means
EM and GMM
10, 11
lecture10
recording
Ismael Martinez and Binulal Narayanan (Fa20)
lecture12-2020.pdf
source
Oct 11 (Tu) Graph Theory
Directed Graphical Models
2
lecture11
recording
Martin Weiss and Eeshan Gunesh Dhekane (Fa18)
lecture11.pdf
source
Oct 14 (Fri) DGM (cont.)
Undirected graphical models
2
lecture12
recording
Philippe Beardsell (Fa18)
lecture12.pdf
source
Oct 18 (Tu) UGM (cont.) 2
lecture13
recording
lecture12 above Hwk 2 due
Hwk 3 out
(hwk 3 source)
Oct 21 (Fri) Inference: elimination alg.
Sum-productr alg.
3, 4
lecture14
recording
lecture13-2018.pdf
source
Sum-product: see lecture16-2020 below
Oct 25 (Tu) Break: look at projects
Nov 1 (Tu) Max-product
Junction tree
HMM
17, 12
lecture15
recording
lecture16-2020.pdf
source
lecture15-2018.pdf
source
Nov 4 (Fri) EM for HMM
Information theory
12, 19
lecture16
recording
lecture15-2018.pdf above
Nov 8 (Tu) Max entropy
MaxENT duality
8
lecture17
(KL geometry: lecture 16 2017)
recording (from 2020)
lecture16-2018.pdf
source
Hwk 3 due
Hwk 4 out
(hwk 4 source)
Project: team formed
Nov 11 (Fri) Exponential families
Estimation in graphical models
8
9
lecture18
recording
MVA lecture5
MVA lecture8
(new scribe notes to come!)
Nov 15 (Tu) MC integration
Sampling
21
(variance reduction: see old lecture18 2017)
lecture19
recording
Nov 18 (Fri) MCMC
Markov chains
Metropolis-Hastings
21
lecture20
recording
Nov 22 (Tu) Gibbs sampling
Variational methods
Bishop: 10.1
lecture21
recording

(skipped parts of: old lecture22 2017 for marginal polytope)
MVA lecture9
Nov 25 (Fri) Gaussian networks

Factor analysis, PCA, CCA
(Kalman filter)
VAE
lecture22
recording

13
old lecture17 Fa2016
14 , (15)
old lecture18 Fa2016
VAE – DL: 20.10.3
MVA lecture6.5
MVA lecture7.3
Nov 29 (Tu) (Guest lecture by Tristan Deleu) Bayesian methods
Model selection

Causality
5, 26
lecture23
recording

Causality for Machine Learning (arXiv 2019)
Elements of Causal Inference (book)
MVA lecture10 Hwk 4 due
Hwk 5 out
(hwk 5 source)
Dec 2 (Fri) (Guest lecture on causality by
Sebastien Lachapelle)
lecture24
recording
Project: 1 page
progress report due
Dec 6 (Tu) Non-parametric models: Gaussian processes
Dirichlet processes
25
lecture25
recording

GP book
DP tutorial
lecture26-2020.pdf
source
Dec 9 (Fri) No lecture: work on hwk 5 or project!
Dec 16 (Fri) Poster presentation
2:00pm-5:00pm
Take-home final out
Dec 23 (Fri) Hwk 5 due
Project report due
Take-home final due