IFT 6269 : Probabilistic Graphical Models - Fall 2018

Description

This course provides a unifying introduction to statistical modeling of multidimensional data through the framework of probabilistic graphical models, together with their associated learning and inference algorithms.

People

Class info

Tentative content

(Detailed outline below)

Evaluation

Prerequisites

Textbook

Homework logistics

Detailed outline (updated often)

Below is a draft detailed outline that will be updated as the class goes on. For now it is the outline recopied from the Fall 2017 version of this class, with the links to the relevant old scribbled notes that will be updated with the new ones gradually. The related chapters in Mike's book are given (but note that they do not exactly correspond with the class content), and also sometimes pointers to the Koller and Friedman book (KF), the Deep Learning book (DL) or the Bishop's book (B). Related past ‘‘scribe notes’’ from the class that I taught in Paris are given for now, and will be updated with the scribe notes as I get them (if I get some).

Date Topics Related chapters
Scribbled notes
Scribe notes Homework milestones
Sept 4 Set-up & overview intro slides
lecture1
Isabela Albuquerque (Fa17)
lecture1.pdf
source
Sept 7 Probability review 2.1.1
DL: 3 (nice and gentle)
KF: 2.1 (more rigorous)
lecture2
William Léchelle (Fa16)
lecture2.pdf
source
Sept 11 Parametric models
Frequentist vs. Bayesian
5
lecture3
Philippe Brouillard and Tristan Deleu (Fa17)
lecture3.pdf
source
Hwk 1 out
(hwk 1 source)
Sept 14 Bayesian (cont.)
Maximum likelihood
lecture4 Philippe Brouillard and Tristan Deleu (Fa17)
lecture4.pdf
Sept 18 MLE (cont.)
Statistical decision theory
1.3 in Bickel & Doksum
Bias-variance tradeoff: 7.3 in Hastie's book
lecture5
Sébastien Lachapelle (Fa17)
lecture5.pdf
source
Sept 21 Properties of estimators lecture6 same as lecture5
Sept 25 Linear regression
Logistic regression
DL: 7.1 (l2, l1-reg.)
6, 7
lecture7
Zakaria Soliman (Fa16)
lecture6.pdf
source
Hwk 1 due
Hwk 2 out
(hwk 2 source)
data.zip
Sept 28 Optimization
Logistic regression (cted) + IRLS
7
DL: 4.3
Boyd's book
lecture8
MVA lecture2
Oct 2 Gen. classification (Fisher)
Derivative tricks for
Gaussian MLE
Kernel trick (skipped)
K-means

Matrix Diff. book
10, 11
old lecture9 (kernel trick)
lecture9
MVA lecture3
Oct 5 GMM and EM 10,11
lecture10
Oct 9 Graph theory
Directed graphical models
2
lecture11
MVA lecture4 Hwk 2 due
Hwk 3 out
(hwk 3 source)
data.zip
Oct 12 DGM (cont.)
Undirected graphical models
2
lecture12
Oct 16 UGM (cont.)
Inference: elimination alg.
3
lecture13
Oct 19 Sum-product alg.
Max-product
junction tree
4, 17
lecture14
MVA lecture7
Oct 23 Break: look at projects
Oct 26 Break: look at projects
Oct 30 HMM and EM 12
lecture15
Hwk 3 due
Hwk 4 out
(hwk 4 source)
Nov 2 Information theory
Max entropy
Duality
19
lecture16
MVA lecture5
Nov 6 MaxENT duality
Exponential families
8
(KL geometry: old lecture16)
lecture17
MVA lecture6
MVA lecture8
Project: team formed
Nov 9 Sampling 21
lecture18
Nov 13 MCMC sampling 21
(variance reduction: see old lecture18)
lecture19
Nov 16
(Gabriel's lecture)
Non-parametric models: Gaussian processes
Dirichlet processes
25
lecture20 slides
GP book
DP tutorial
Nov 20 MCMC (cont.)
Gibbs sampling
lecture21 Hwk 4 due
Hwk 5 out
(hwk 5 source)
Nov 23 Variational methods
Estimation in graphical models
Bishop: 10.1
lecture22
9
(skipped parts of: old lecture22)
Nov 27 Bayesian methods
Model selection
5, 26
lecture23
MVA lecture10 Project: 1 page
progress report due
Nov 30 Gaussian networks

Factor analysis, PCA, CCA
(Kalman filter)
VAE
old lecture24

13
old lecture17 Fa2016
14 , (15)
old lecture18 Fa2016
VAE – DL: 20.10.3
Dec 4 No lecture this week (NIPS)
work on your project!
Dec 11 Poster presentation
1:00pm-4:00pm
mezzanine of Jean-Coutu atrium
Hwk 5 due
Take-home final out
Dec 19 Project report due
Take-home final due