IFT 6268, Fall 2013

Machine Learning for Vision

Time and Place

Tuesdays 1:30pm - 3:30pm
Z-205 Pav. Claire-McNicoll

Wednesdays 11:30am - 1:30pm
Z-300 Pav. Claire-McNicoll


Roland Memisevic
Office : 3349, Pav. Andre-Aisenstadt
Office hours: drop in or by appointment.
email: memisevr@iro.umontreal.ca


Machine learning and visual perception; natural image statistics; learning visual features; bio-inspired models; learning mid-level features, motion, structure; vision and "big data".

Course description

Machine learning has made huge progress in recent years in building vision systems from the bottom up: by utilizing large amounts of image data and a minimum of hand-tweaking or human intervention. Most of the recent advances in visual perception are due to the fact that images are not "random": The set of natural images (imagine all photos that one could ever take on earth) is much smaller than the vast space of all possible images (all possible configurations of colored pixels). Natural images, for example, tend to contain large, homogeneous areas, separated by few edges and fewer junctions. The combination of these features gives rise to objects, people, landscapes, and other things we typically see and care about. By learning to utilize this inherent statistical structure in images, data-driven learning makes it possible to simplify the task of making sense of images, and to translate advances in compute power into advances in building vision systems.

In this course we will survey recent research on machine learning of visual perception, paying particular attention to bottom-up, data-driven methods and the statistics of natural images. The format of the course will be a mix of lectures and discussions of recent papers in this emerging field. Final projects will be research based and may ultimately lead to a research paper in this area.


Familiarity with calculus, linear algebra and statistics is required. A background in vision is not required. Some experience with machine learning, for example, as taught in IFT6141 (Reconnaissance des formes) or IFT6390 (Fondements de l'apprentissage machine) will be useful. If unsure about the sufficiency of your background, contact instructor.


Although there is no official class textbook, we will make some use of some material from "Natural Image Statistics: A Probabilistic Approach to Early Computational Vision" by Hyvarinen, Hurri, Hoyer (Springer 2009).

Marking scheme

Time Table

Date Topic Readings Notes
Introduction / background
Sept. 4 Lecture: Introduction notes
Sept. 10 Probability/linear algebra review
Sept. 11 Python review
Learning about images
Other readings
  • Some Informational Aspects Of Visual Perception. F. Attneave. Psychological Review 1954. [pdf]
  • Deconvolutional Networks. M. Zeiler, D. Krishnan, G. Taylor, R. Fergus. CVPR 2010. [pdf]
  • Fields of Experts. S. Roth, M. Black. IJCV 2007. [pdf]
  • How to generate realistic images using gated MRF's. M.A. Ranzato, V. Mnih, G. Hinton. NIPS 2010. [pdf]
Sept. 17 Filtering, convolution, phasor signals Reading 1: Possible Principles Underlying the Transformations of Sensory Messages. (Barlow, 1961) [pdf]
Sept. 18 Fourier transforms on images notes
Sept. 24 Sampling, leakage, windowing Reading 2: sparse coding (Foldiak, Endres; scholarpedia 2008) Reading 1 due
Sept. 25 PCA, stationarity and Fourier bases notes
Assignment1 (worth 10%)
Oct. 1 PCA and whitening cont'd. Reading 3: Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images. (Olshausen, Field 1996). [pdf] Reading 2 due
Oct. 2 Some facts about vision in the brain Assignment 1 due
solution to question 3
solution to question 4
minimize.py (required if using Logreg.train_cg)
Oct. 8 Vision in the brain cont'd. Reading 4: The ``independent components'' of natural scenes are edge filters. (Bell, Sejnowski 1997) [pdf]
Reading 3 due
Oct. 9 Whitening vs independence, ICA Assignment2 (worth 10%)
Oct. 15 ICA cont'd. Reading 5: Review backprop (aka error backpropagation). For example, by watching week 5 of this course or by reading Bishop, chapter 5.
If/once you know backprop, check out theano and these tutorials.
Reading 4 due
Oct. 16 Overcomplete codes, energy based models Assignment 2 due
solution to questions 3 and 4
Oct. 29 Restricted Boltzmann machines Reading 6: Feature Discovery by Competitive Learning (Rumelhart, Zipser; 1985) [pdf] (email instructor if you cannot access the pdf) notes
Reading 5 due
Oct. 30 Competitive Hebbian learning, k-means notes
Nov. 5 Autoencoders, Convolutional networks Reading 7: Pick one of the papers we will discuss on Nov 6 and Nov 12 (other than the one you present) and read it very carefully. In your email, say which one you read.
Reading 6 due
Nov. 6 Paper presentations, discussion
  • Building high-level features using large scale unsupervised learning. Q. Le, M.A. Ranzato, R. Monga, M. Devin, K. Chen, G. Corrado, J. Dean, A. Ng. ICML 2012. [pdf] (David K.)
  • An analysis of single-layer networks in unsupervised feature learning. A. Coates, H. Lee, A. Y. Ng. AISTATS 2011. [pdf] (Gabriel F.)
Assignment3 (worth 10%)
Nov. 12 Paper presentations, discussion Reading 8: Spatiotemporal energy models for the perception of motion. Adelson and Bergen, 1985 [pdf]
  • Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations. H. Lee, et al. ICML 2009. [pdf] (Eugene V.)
  • ImageNet Classification with Deep Convolutional Neural Networks. A. Krizhevsky, I. Sutskever, G.E. Hinton. NIPS 2012. [pdf] (Sebastien O.)
Reading 7 due
Learning about motion, geometry, invariance, shape
Other readings
  • Neural Encoding of Binocular Disparity: Energy Models, Position Shifts and Phase Shifts. D. Fleet, H. Wagner, D. Heeger. Vision Research 1996. [pdf]
  • Convolutional Learning of Spatio-temporal Features. G. Taylor, et al. ECCV 2010 [pdf]
  • Learning Intermediate-Level Representations of Form and Motion from Natural Movies. C. Cadieu, B. Olshausen. Neural Computation 2012. [pdf]
  • On Random Weights and Unsupervised Feature Learning. A Saxe, et al. ICML 2011 [pdf]
  • Transforming Auto-encoders. G. Hinton, et al. ICANN 2011. [pdf]
  • Compete to compute Srivastave et al. NIPS 2013 [pdf]
Nov. 13 Learning about relations Assignment 3 due
solution to question 2
slides from my cifar tutorial
Nov. 19 Paper presentations, discussion Reading 9: Pick one of the papers we will discuss on Nov 19 and Nov 20 (other than the one you present) and read it very carefully. In your email, say which one you read.

DeViSE: A Deep Visual-Semantic Embedding Model Frome et al. NIPS 2013 [pdf] (Sebastien O.)
Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. Le, et al. CVPR 2011 [pdf] (David K.)
Reading 8 due
Nov. 20 Paper presentations, discussion Deep Learning of Invariant Features via Simulated Fixations in Video. Zou et al. NIPS 2012. [pdf] (Gabriel F.)
Learning to combine foveal glimpses with a third-order Boltzmann machine. H. Larochelle and G. Hinton. NIPS 2010. [pdf] (Eugene V.)
Nov. 26 Learning about relations cont'd. Reading 10: Pick a highly relevant related work for your class project, and read it very carefully. In your email, say which one you read. (due Dec. 3)

Reading 9 due
Nov. 27 Project discussions
Nov. 28, 12:30pm
Location: Pav. Andre-Aisendstadt, rm 3195
Structured prediction, misc, wrap-up notes

Other topics we may or may not have the time to cover include saliency, attention, detailed scene understanding, multi-modal learning.