next up previous
Next: Hybrids of Discrete and Up: Markovian Models for Sequential Previous: Variable Length Markov Models

State Space Models

 

In this section we draw a few connections between HMMs (which traditionally are based on a discrete hidden state) and state space models, which can be seen as HMMs with a continuous vector state variable .

To keep the mathematics tractable, most state space models are restricted to a transition model which is Gaussian with a mean vector that is a linear function of the previous state (and possibly of the current inputs, for input/output models):

where is the probability of observing vector v under a Gaussian distribution with mean and covariance matrix . A and B are matrices which are parameters of the model. Various models for the covariance have been proposed: it may be constant, or it may depend on the previous state and the current input. Like the Markov switching models introduced earlier, state space models are more generally expressed functionally:

where is a zero-mean Gaussian random variable. Similarly, a Gaussian emission model can be expressed as in equation 12.

The Kalman filter [&make_named_href('', "node31.html#Kalman61","[119]")] is in fact such a model, and the associated algorithms allow to compute in a forward recursion (thus solving the filtering problem). Similarly to Markov switching models, a backward recursion (the Rauch equations [&make_named_href('', "node31.html#Rauch63","[120]")]) allows to compute the posterior probabilities for T > t (thus solving the smoothing problem).

In the context of real-time control and other applications (such as tracking [&make_named_href('', "node31.html#Bar-Shalom78","[121]"), &make_named_href('', "node31.html#Bar-Shalom93","[122]")]) where learning must be on-line, numerical maximization of the likelihood can be performed recursively with a second-order method which requires only gradients [&make_named_href('', "node31.html#Ljung+Soderstrom83","[123]")]. For off-line applications, the EM algorithm can also be used [&make_named_href('', "node31.html#Zoubin96","[124]")], with a backward pass that is equivalent to the Rauch equations.





Yoshua Bengio
Tue Oct 7 08:34:36 EDT 1997