In this section we draw a few connections between HMMs
(which traditionally are based on a discrete hidden state)
and state space models, which can be seen as HMMs with
a continuous vector state variable
.
To keep the mathematics tractable, most state space models are restricted to a transition model which is Gaussian with a mean vector that is a linear function of the previous state (and possibly of the current inputs, for input/output models):
where
is the probability of observing
vector v under a Gaussian distribution with mean
and covariance matrix
. A and B are matrices
which are parameters of the model. Various models
for the covariance
have been proposed: it may be constant,
or it may depend on the previous state and the current input.
Like the Markov switching
models introduced earlier, state space models are
more generally expressed functionally:
where
is a zero-mean Gaussian random variable.
Similarly, a Gaussian emission model can be expressed as
in equation 12.
The Kalman filter [&make_named_href('',
"node31.html#Kalman61","[119]")] is in fact such a model,
and the associated algorithms allow to compute
in a forward recursion (thus solving the filtering
problem). Similarly to Markov switching models,
a backward recursion (the Rauch equations [&make_named_href('',
"node31.html#Rauch63","[120]")]) allows to
compute the posterior probabilities
for T > t (thus solving the smoothing problem).
In the context of real-time control and other applications (such as tracking [&make_named_href('', "node31.html#Bar-Shalom78","[121]"), &make_named_href('', "node31.html#Bar-Shalom93","[122]")]) where learning must be on-line, numerical maximization of the likelihood can be performed recursively with a second-order method which requires only gradients [&make_named_href('', "node31.html#Ljung+Soderstrom83","[123]")]. For off-line applications, the EM algorithm can also be used [&make_named_href('', "node31.html#Zoubin96","[124]")], with a backward pass that is equivalent to the Rauch equations.