Modelling high-dimensional sequential data
using distributed hidden state
par/by Geoffrey Hinton
University of Toronto
and
Canadian Institute for Advanced Research
I will describe a new model for sequential data. The model uses a large
number of hidden binary variables that are symmetrically connected to the
observable variables to form a restricted Boltzmann machine. In addition,
the hidden variables are linked together by directed connections to form a
sigmoid belief net. I will describe how the model is trained and show that
it can learn a good generative model of human motion from motion capture
data. We will show that the same type of model is very good at predicting
the next word in a sentence and we will describe ways in which the model can
be improved.
Joint work with Graham Taylor, Andriy Mnih and Ilya Sutskever