IFT 6269 : Probabilistic Graphical Models - Fall 2024

Last year version: Fall 2023

Description

This course provides a unifying introduction to statistical modeling of multidimensional data through the framework of probabilistic graphical models, together with their associated learning and inference algorithms.

People

Class info (hybrid lectures)

The lectures are taught in hybrid mode: the lectures will be given at the Mila Auditorium 1 and 2, 6650 rue Saint-Urbain, but also connected synchronously on Zoom (see info on Studium). They will also be recorded for further review or for those in remote time zones.

Tentative content

(Detailed outline below)

Evaluation

Prerequisites

Textbook

Homework logistics

Detailed outline (updated often)

Below is a draft detailed outline that will be updated as the class goes on. For now it is the outline recopied from the Fall 2023 version of this class, with the links to the relevant old scribbled notes that will be updated with the new ones gradually. The related chapters in Mike's book are given (but note that they do not exactly correspond with the class content), and also sometimes pointers to the Koller and Friedman book (KF), the Deep Learning book (DL) or the Bishop's book (B). Related past ‘‘scribe notes’’ from the previous instantiations of this class as well as (for later lectures) the class that I taught in Paris are given for now, and will be updated with the more updated scribe notes if needed (and I get some).

Date Topics Related chapters
Scribbled notes
Recordings
Scribe notes Homework milestones
Sept 5 (Thurs) Set-up & overview lecture1
recording
Isabela Albuquerque (Fa17)
lecture1.pdf
source
Sept 9 (Mon) Probability review 2.1.1
DL: 3 (nice and gentle)
KF: 2.1 (more rigorous)
lecture2
recording
William Léchelle (Fa16)
lecture2.pdf
source
Sept 12 (Thurs) Prob. review (cont.)
Parametric models
5
lecture3
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture3.pdf
source
Hwk 1 out
(hwk 1 source)
Sept 16 (Mon) Frequentist vs. Bayesian
Maximum likelihood
lecture4
recording
Philippe Brouillard and Tristan Deleu (Fa17)
lecture4.pdf
source
Sept 19 (Thurs) MLE (cont.)
Statistical decision theory
1.3 in Bickel & Doksum
Bias-variance tradeoff: 7.3 in Hastie's book
lecture5
recording
Sébastien Lachapelle (Fa17)
lecture5.pdf
source
Sept 23 (Mon) Properties of estimators
and MLE
lecture6
recording
same as lecture5
Sept 26 (Thurs) Linear regression
Logistic regression
DL: 7.1 (l2, l1-reg.)
6, 7
lecture7
recording (only 1 hour!)
old recording 2023
Zakaria Soliman (Fa16)
lecture6.pdf
source
Hwk 1 due
Hwk 2 out
Sept 30 (Mon) Numerical Optimization
Logistic regression (cted) and IRLS

7
DL: 4.3
Boyd's book
lecture8
recording
Eeshan Gunesh Dhekane and Younes Driouiche (Fa18)
lecture8.pdf
source
Oct 3 (Thurs) Gen. classification (Fisher)
Derivative tricks for
Gaussian MLE

Kernel trick (skipped)
7
Matrix Diff. book
lecture9

recording

lecture9 2017 (kernel trick) (skipped)
Eeshan Gunesh Dhekane (Fa18)
lecture9.pdf
source
Oct 7 (Mon) latent variable model
K-means
EM and GMM
10, 11
lecture10
recording
Ismael Martinez and Binulal Narayanan (Fa20)
lecture12-2020.pdf
source
Oct 10 (Thurs) Graph Theory
Directed Graphical Models
2
lecture11
recording
Martin Weiss and Eeshan Gunesh Dhekane (Fa18)
lecture11.pdf
source
Oct 14 (Mon) Thanksgiving holiday
Oct 17 (Thurs) DGM (cont.)
Undirected graphical models
2
lecture12
recording
Philippe Beardsell (Fa18)
lecture12.pdf
source
Hwk 2 due
Hwk 3 out
(hwk 3 source)
Oct 21 (Mon) Break: look at projects
Oct 28 (Mon) UGM (cont.) 2
lecture13
recording
Oct 31 (Thurs) Inference: elimination alg.
Sum-product alg.
3, 4
old lecture14
old recording
lecture13-2018.pdf
source
Sum-product: see lecture16-2020 below
Nov 4 (Mon) Max-product
Junction tree
HMM
no live lecture: recording only
17, 12
old lecture15
old recording
lecture16-2020.pdf
source
lecture15-2018.pdf
source
Nov 7 (Thurs) EM for HMM
Information theory
no live lecture: recording only
12, 19
old lecture16
old recording
lecture15-2018.pdf above Hwk 3 due
Hwk 4 out
Project: team formed
Nov 11 (Mon) Max entropy
MaxENT duality
8
old lecture17
(KL geometry: lecture 16 2017)
old recording
lecture16-2018.pdf
source
Nov 14 (Thurs) Exponential families
Estimation in graphical models
8
9
old lecture18
old recording
lecture18-2021.pdf
source
Nov 18 (Mon) MC integration
Sampling
21
(variance reduction: see old lecture18 2017)
old lecture19
old recording
(new scribe notes to come!)
Nov 21 (Thurs) MCMC
Markov chains
Metropolis-Hastings
21
old lecture20
old recording
Nov 25 (Mon) Gibbs sampling
Variational methods
Bishop: 10.1
old lecture21
old recording

(skipped parts of: old lecture22 2017 for marginal polytope)
MVA lecture9
Nov 28 (Thurs) Bayesian methods
Model selection

Causality
5, 26
old lecture22
old recording

Causality for Machine Learning (arXiv 2019)
Elements of Causal Inference (book)
MVA lecture10 Hwk 4 due
Hwk 5 out
Dec 2 (Mon) Guest lecture on GFlowNets by Juan and Mansi? old lecture23
old recording
Project: 1 page
progress report due
Dec 5 (Thurs) Gaussian networks

Factor analysis, PCA, CCA
(Kalman filter)
VAE
old lecture24
old recording

13
old lecture17 Fa2016
14 , (15)
old lecture18 Fa2016
VAE – DL: 20.10.3
MVA lecture6.5
MVA lecture7.3
(skipped this year) (Guest lecture on causality by
Sebastien Lachapelle)
old lecture24
old recording
(skipped this year) Non-parametric models: Gaussian processes
Dirichlet processes
25
old lecture25
old recording

GP book
DP tutorial
lecture26-2020.pdf
source
Dec 9 (Mon) No lecture this week (NeurIPS)
work on hwk 5 or your project!
Dec 16 (Mon) Take-home final out
Dec 19 (Thurs) Poster presentation
1:30pm-4:30pm
Dec 23 (Mon) Hwk 5 due
Project report due
Take-home final due