Modelling Image Patches With A Directed Hierarchy Of Markov Random Fields

par/by Simon Osindero
Department of Computer Science
University of Toronto

I will describe an efficient learning procedure for a type of multilayer generative model that combines the best aspects of Markov Random Fields (MRF's) and deep, directed belief nets (DBN's). In particular, I will consider hierarchies in which each hidden layer has its own MRF whose energy function is modulated by the top-down directed connections from the layer above. Our proposed algorithm allows these generative models to be learned one layer at a time, and when learning is complete we are able to use a fast and simple inference procedure for computing a good approximation to the posterior distribution on all of the hidden layers. I will present preliminary results of our approach applied to an image-patch dataset, and will show that this type of model is good at capturing the statistics of patches of natural scenes. This is joint work with Geoff Hinton.