Deep belief networks are universal approximators and a recurrent neural network that learns to remember
par/by Ilya Sutskever
Department of Computer Science
University of Toronto
The talk will consist of two parts. In the first part I will prove that narrow deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, and describe a simple greedy algorithm that can learn such networks in an impractical manner.
In the second part I will introduce a new kind of recurrent neural networks that can learn long term dependencies much better than standard recurrent neural networks.