Hyatt hotel, Vancouver, Canada, December 6th, 2007, 2pm-5:30pm, Georgia B room, 2nd floor
Organizers: Yoshua Bengio, Yann LeCun, Ruslan Salakhutdinov and Hugo Larochelle
This is the afternoon part of NIPS'2007 special Neuro-Thursday, sponsored by the Canadian Institute For Advanced Research (CIFAR).
Theoretical results strongly suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need "deep architectures", which are composed of multiple levels of non-linear operations (such as in neural nets with many hidden layers). Searching the parameter space of deep architectures is a difficult optimization task, but learning algorithms (e.g. Deep Belief Networks) have recently been proposed to tackle this problem with notable success, beating the state-of-the-art in certain areas.
This workshop is intended to bring together researchers interested in the question of deep learning in order to review the current algorithms' principles and successes, but also to identify the challenges, and to formulate promising directions of investigation. Besides the algorithms themselves, there are many fundamental questions that need to be addressed: What would be a good formalization of deep learning? What new ideas could be exploited to make further inroads to that difficult optimization problem? What makes a good high-level representation or abstraction? What type of problem is deep learning appropriate for?
The workshop is also going to be an opportunity to celebrate Geoff Hinton's 60th birthday and his contributions to theories of neural information processing.
Registration is required in order to attend the workshop. Free registration for an evening bus going to Whisler is also offered to workshop participants. This bus trip is different from the one offered on the NIPS registration page (i.e. you do not need to ask for it when registering to NIPS).
The number of places at the Hyatt is limited, so early registration is highly recommended!
The registration page can be found here.
Videos of all workshop talks are available here.
2:00pm - 2:05pm Introductory remarks |avi|
2:05pm - 2:25pm Yee-Whye Teh, Gatsby Unit : Setting the Stage: Complementary Prior and Variational Bounds |pdf part1.avi part2.avi|
2:25pm - 2:45pm John Langford, Yahoo Research: (Lack of) Deep Learning Theory |pdf avi|
2:45pm - 3:05pm Yoshua Bengio, University of Montreal: Optimizing Deep Architectures |pdf part1.avi part2.avi|
3:05pm - 3:25pm Yann Le Cun, New York University: Learning a Deep Hierarchy of Sparse Invariant Features |pdf part1.avi part2.avi|
3:25pm - 3:45pm Martin Szummer, Microsoft Research: Deep Networks for Information Retrieval |pdf part1.avi part2.avi|
3:45pm - 4:00pm Coffee break
4:00pm - 4:20pm Max Welling, University of California: (Infinite) Deep Networks |ppt part1.avi part2.avi|
4:20pm - 4:40pm Rajat Raina, Stanford University: Self-taught Learning |ppt pptx avi|
4:40pm - 5:00pm Geoff Hinton, University of Toronto: How to do Backpropagation in a Brain |ppt avi|
5:00pm - 5:30pm Discussion |avi|
First paper introducing Deep Belief Networks (as generative models):
Review paper on deep architectures and details of Deep Belief Nets and Restricted Boltzmann Machines:
Book chapter about the philosophy behind deep architecture model, motivating them in the context of Artificial Intelligence
Deep Belief Networks as a simple way of initializing a deep feed-forward neural network:
General study of the framework of initializing a deep feed-forward neural network using a greedy layer-wise procedure:
An application of greedy layer-wise learning of a deep autoassociator for dimensionality reduction:
A way to use the greedy layer-wise learning procedure to learn a useful embeding for k nearest neighbor classification:
Different theoretical results about Restricted Boltzmann Machines (RBMs) and Deep Belief Networks, like the universal approximation property of RBMs:
A novel way of using greedy layer-wise learning for Convolutional Networks:
How to generalize Restricted Boltzmann Machines to types of data other than binary using exponential familly distribution:
An evaluation of deep networks on many datasets related to vision:
Application of deep learning in the context of information retrieval:
I | Attachment | Action | Size | Date | Who | Comment |
---|---|---|---|---|---|---|
deep_learning_bengio.pdf | manage | 430.4 K | 15 Jan 2008 - 16:02 | HugoLarochelle | ||
ppt | deep_learning_hinton.ppt | manage | 254.5 K | 15 Jan 2008 - 16:03 | HugoLarochelle | |
deep_learning_langford.pdf | manage | 54.2 K | 15 Jan 2008 - 16:03 | HugoLarochelle | ||
deep_learning_lecun.pdf | manage | 8713.6 K | 15 Jan 2008 - 16:04 | HugoLarochelle | ||
ppt | deep_learning_raina.ppt | manage | 2287.0 K | 15 Jan 2008 - 16:04 | HugoLarochelle | |
pptx | deep_learning_raina.pptx | manage | 1325.7 K | 15 Jan 2008 - 16:04 | HugoLarochelle | |
deep_learning_szummer.pdf | manage | 512.7 K | 21 Jan 2008 - 10:49 | HugoLarochelle | ||
deep_learning_teh.pdf | manage | 278.5 K | 15 Jan 2008 - 16:04 | HugoLarochelle | ||
ppt | deep_learning_welling.ppt | manage | 1174.5 K | 15 Jan 2008 - 16:05 | HugoLarochelle |