Learning about images 
Suggested readings

[Background]
Some Informational Aspects Of Visual Perception. F. Attneave. Psychological Review 1954. [pdf]

[Background]
Possible Principles Underlying the Transformations of Sensory Messages. H.B. Barlow. Sensory communication 1961. [pdf]

[Background] Emergence of SimpleCell Receptive Field Properties by Learning a Sparse Code for Natural Images. B.A. Olshausen, D.J. Field. Nature, 1996. [pdf]

[Background] The `Independent Components' of Natural Scenes are Edge Filters. T. Bell, T. Sejnowski. Vis. Res. 1997 [pdf]

[Background]
Independent Component Analysis: Algorithms and Applications. A. Hyvarinen, E. Oja. Neural Computation 2000. [pdf]

[Background] Sparse Coding. P. Foldiak, D. Endres. Scholarpedia. [link]

Do We Know What the Early Visual System Does? M. Carrandini, et al. The Journal of Neuroscience 2005. [pdf].

Slow Feature Analysis: Unsupervised Learning of Invariances. L. Wiskott, T. Sejnowski. Neural Computation 2002. [pdf]

Natural Images, Gaussian Mixtures and Dead Leaves. D. Zoran, Y. Weiss. NIPS 2012 [pdf]

Date

Topic

Readings

Materials

Jan. 22

Lecture: Introduction


notes

Jan. 25

Lecture: Review of basic statistics, linear algebra


notes
Python logsumexp implementation.

Jan. 29

Lecture: Fourier representation and frequency analysis

 useful reading: textbook chapters 2, 20.

notes

Jan. 31

Lecture: Fourier part II


notes

Feb. 5

Lecture: Fourier part III, Preprocessing, PCA, whitening

 Possible Principles Underlying the Transformations of Sensory Messages. H.B. Barlow. [pdf]
 Interesting, not required: Some Informational Aspects Of Visual Perception. F. Attneave. [pdf]
 Useful background on PCA: textbook, sections 5.1  5.9.2.

assignment 1 (worth 10%)
notes

Feb. 8

Lecture: PCA and Fourier representation,
some biological aspects of image processing

 Textbook chapter 3.
 Sparse Coding. P. Foldiak, D. Endres. Scholarpedia. [link]

notes

Feb. 12

Lecture: ICA, sparse coding

 useful background reading:
The `Independent Components' of Natural Scenes are Edge Filters. T. Bell, T. Sejnowski. Vis. Res. 1997 [pdf]

assignment 1 due at the beginning of class
notes

Feb. 15

Lecture: Feature Learning II

 read the presentation guidelines

Emergence of SimpleCell Receptive Field Properties by Learning a Sparse Code for Natural Images. B.A. Olshausen, D.J. Field. Nature, 1996. [pdf]

notes

Feb. 19

Lecture: Feature Learning III
review of assignment 1
Presentations/Discussion

 An analysis of singlelayer networks in unsupervised feature learning. A. Coates, H. Lee, A. Y. Ng. AISTATS 2011. [pdf] (Salah R.)

assignment 2 (worth 10%)
A1 solution Q2
A1 solution Q4
notes

Feb. 22

Lecture: More on kmeans, energy based models
Presentations/Discussion


Building highlevel features using large scale unsupervised learning. Q. Le, M.A. Ranzato, R. Monga, M. Devin, K. Chen, G. Corrado, J. Dean, A. Ng. ICML 2012. [pdf] (Pierre Luc C.)

ImageNet Classification with Deep Convolutional Neural Networks. A. Krizhevsky, I. Sutskever, G.E. Hinton. NIPS 2012. [pdf]

notes

Feb. 26

Lecture: Restricted Boltmann Machines
Presentations/Discussion


Noisecontrastive estimation: A new estimation principle for unnormalized statistical models. M. Gutmann, A. Hyvarinen. AISTATS 2010. [pdf] (Sebastien J.)
 Sparse filtering. J. Ngiam, et. al. NIPS 2011. [pdf] (Antoine M.)

assignment 2 due at the beginning of class
notes

Mar. 1

Lecture: more on RBMs
MRF models
Presentations/Discussion


Fields of Experts. S. Roth, M. Black. IJCV 2007. [pdf] (Francis Q. L.)

Deconvolutional Networks. M. Zeiler, D. Krishnan, G. Taylor, R. Fergus. CVPR 2010. [pdf] (Vincent A.)


Learning about motion, geometry and invariance 
Suggested readings

[Background] Spatiotemporal energy models for the perception of motion. E.H. Adelson and J.R. Bergen. Journal Opt. Soc. Am. [pdf]

[Background] Neural Encoding of Binocular Disparity: Energy Models, Position Shifts and Phase Shifts. D. Fleet, H. Wagner, D. Heeger. Vision Research 1996. [pdf]

[Background] Learning Invariance from Transformation Sequences. P. Foldiak. Neural Computation 1991. [pdf].

Topographic Independent Component Analysis. A. Hyvarinen, P.O. Hoyer, M. Inki. Neural Computation 2001. [pdf]

Unsupervised learning of image transformations. R. Memisevic and G.E. Hinton. CVPR 2007. [pdf]

A multilayer sparse coding network learns contour coding from natural images. P.O. Hoyer and A. Hyvarinen. Vision Research 2002. [ps]

Emergence of Phase and ShiftInvariant Features by Decomposition of Natural Images into Independent Feature Subspaces. A. Hyvarinen, P. Hoyer. Neural Computation 2000. [pdf]

Dynamic Scene Understanding: The Role of Orientation Features in Space and Time in Scene Classification. K.G. Derpanis, M. Lecce, K. Daniildis, R.P. Wildes. CVPR 2012. [pdf]

Action spotting and recognition based on a spatiotemporal orientation analysis. K.G. Derpanis, M. Sizintsev, K. Cannons, R.P. Wildes. PAMI 2013. [pdf]

Date

Topic

Readings

Materials

Mar. 12

review of assignment 2
brief review of squaring and complex cells

Spatiotemporal energy models for the perception of motion. E.H. Adelson and J.R. Bergen. Journal Opt. Soc. Am. [pdf]

assignment 3 (worth 10%)
A2 solution Q3 (online kmeans)
A2 solution Q3 (kmeans on cifar)

Mar. 15

Lecture: Relations, gating and complex cells I

Learning to relate images. Roland Memisevic TPAMI 2013 [pdf]

notes

Mar. 19

Lecture: Relations, gating and complex cells II


assignment 3 due at the beginning of class

Mar. 22

Presentations/Discussion
Energy models on single images


What is the Best MultiStage Architecture for Object Recognition?
K. Jarrett, K. Kavukcuoglu, M.A. Ranzato, Y. LeCun. CVPR 2009. [pdf] (Nicholas L.)

On Random Weights and Unsupervised Feature Learning. A Saxe, et al. ICML 2011. [pdf] (Pierre Luc C.)

A3 solution Q3

Mar. 26

Discussion of assignment 3 Q3
Lecture: Relations, gating and complex cells III


The slides from my IPAM tutorial:
part 1 and
part 2.

Apr. 2

Discussion of assignment 3 Q1/Q2
Presentations/Discussion
Energy models on single images II


How to generate realistic images using gated MRF's. M.A. Ranzato, V. Mnih, G. Hinton. NIPS 2010. [pdf] (Francis Q.L.)


Apr. 5

Presentations/Discussion
Motion features and activity recognition


Convolutional Learning of Spatiotemporal Features. G. Taylor, R. Fergus, Y. LeCun and C. Bregler. ECCV, 2010. [pdf] (Xavier B.)

Learning hierarchical invariant spatiotemporal features for action recognition with independent subspace analysis. Q.V. Le, W.Y. Zou, S.Y. Yeung, A.Y. Ng. CVPR 2011 [pdf] (Antoine M.)


Apr. 9

Presentations/Discussion
Learning about images from movies


Learning IntermediateLevel Representations of Form and Motion from Natural Movies. C.F. Cadieu, B.A. Olshausen. Neural Computation 2012. [pdf] (Xavier B.)

Deep Learning of Invariant Features via Simulated Fixations in Video. W. Zou, A. Ng, S. Zhu, K. Yu. NIPS 2012. (Sebastien J.)
[pdf]


Apr. 12

Presentations/Discussion
Fixations
Group structure and topography


Topographic Independent Component Analysis. A. Hyvarinen, P.O. Hoyer, M. Inki. Neural Computation 2001. [pdf]

Learning Invariant Features through Topographic Filter Maps. K. Kavukcuoglu, M.A. Ranzato, R. Fergus, Y. LeCun. CVPR 2009. [pdf] (Magatte D.)

Learning to combine foveal glimpses with a thirdorder Boltzmann machine. H. Larochelle and G. Hinton. NIPS 2010. [pdf] (Vincent A.)


Learning about shape, learning where to look, miscellaneous. 
Suggested readings

A Model of SaliencyBased Visual Attention for Rapid Scene Analysis. L. Itti, C. Koch, E. Niebur. PAMI 1998. [pdf]
 Searching for objects driven by context. Alexe, et al. NIPS 2012 [pdf]

Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. ICML 2009. [pdf]

Learning where to attend with deep architectures for image tracking. M. Denil, L. Bazzani, H. Larochelle, N. de Freitas. Neural Computation 2012. [pdf]

Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style. G. Taylor, G. Hinton. ICML 2009. [pdf]

Learning a Generative Model of Images by Factoring Appearance and Shape. N. Le Roux, N. Heess, J. Shotton, J. Winn. Neural Computation 2012. [pdf]

Date

Topic

Readings

Materials

Apr. 16

Presentations/Discussion
Miscellaneous topics

 WSABIE: Scaling Up To Large Vocabulary Image Annotation. J. Weston, et al. IJCAI [pdf] (Salah R.)
 The Shape Boltzmann Machine: a Strong Model of Object Shape. S.M.A.Eslami, N.Hess, J.Winn. CVPR 2012. [pdf] (Nicholas L.)
 Emergence of ObjectSelective Features in Unsupervised Feature Learning. A. Coates et al. NIPS 2012. [pdf] (Magatte D.)


Apr. 19

Wrapup
Final Project Presentations and Discussions

 Transforming Autoencoders. G. Hinton, A. Krizhevsky, S. Wang. ICANN 2011. [pdf]
 Project Nicholas L.
 Project Francis Q. L.
 Project Antoine M.


Apr. 22

Project office hour



Apr. 23

Final Project Presentations and Discussions

 Project Sebastien J.
 Project Xavier B.
 Project Vincent A.B.
 Project Pierre Luc C.
 Project Salah R.
 Project Magatte D.


Apr. 30


Final Projects Due

