next up previous
Next: Markov Switching Models Up: Markovian Models for Sequential Previous: Integrating Artificial Neural Networks

Input-Output HMMs

 

Input-Output Hidden Markov Models (IOHMMs) [&make_named_href('', "node31.html#Bengio+Frasconi-nips7-iohmms","[5]")] (or Conditional HMMs) are simply HMMs for which the emission and transition distributions are conditional on another sequential random variable, called the input sequence, and that we will note . In that case, the observations modeled with the emission distributions are called outputs, and the model represents not the distribution of sequences but instead the conditional distribution . In the simpler models first presented here, the input and output sequences have the same length, but a recent extension [&make_named_href('', "node31.html#Bengio+Bengio96","[93]")] (section 5.3) allows input and output sequences of different lengths. Transducers (section 6) which can be seen as generalizations of such conditional distributions, also allow input and output sequences of different lengths. The conditional independence assumption of a synchronous IOHMM are represented in the probabilistic graphical model of Figure 10.

  
Figure 10: Probabilistic graphical model representing the independence assumptions of a synchronous Input-Output Hidden Markov Model. The state sequence is , the output sequence is , and the input sequence is .

In the simpler case in which the input and output sequences are synchronous, the mathematics of IOHMMs are very similar to that of HMMs but more general. For this reason we will explain the EM algorithm used to train HMMs (and IOHMMs) in this section (in subsection 5.2). Whereas in ordinary HMMs the emission distributions are given by a homogeneous model , in IOHMMs, they are given by a time-varying conditional model (i.e., the emission density associated to each state changes in function of the input ). Similarly, instead of time-invariant transition probabilities , in IOHMMs we have . More generally, values of the inputs at different time steps around can be used to condition these distributions. The ``forward'' phase copmutation in IOHMMs is identical in form to that in HMMs:

 

Whereas HMMs used for pattern recognition are often trained by choosing parameters to maximize the likelihood of the acoustic observations given the correct classification sequence, , IOHMMs may be trained to maximize the likelihood of decision variables given the actually observed (e.g., acoustic) variables .

In the literature on learning algorithms [&make_named_href('', "node31.html#Cacciatore-nips94","[4]"), &make_named_href('', "node31.html#Bengio+Frasconi-nips7-iohmms","[5]"), &make_named_href('', "node31.html#Meila96","[6]")], IOHMMs have been proposed for sequence processing tasks, with complex emission and transition models based on ANNs. There are also earlier treatments of similar models in statistics [&make_named_href('', "node31.html#Lindgren78","[94]")] and econometrics [&make_named_href('', "node31.html#Hamilton89","[12]")]. In the control and reinforcement learning literature, similar models have been called Partially Observable Markov Decision Processes [&make_named_href('', "node31.html#Sondik73","[95]"), &make_named_href('', "node31.html#Sondik78","[96]"), &make_named_href('', "node31.html#Chrisman92AAAI","[3]")]. In this case, the objective is not to model an output sequence given an input sequence, but rather, to find the input sequence (in fact a sequence of actions) which minimizes a cost function defined on the sequence of outputs (which are observed). In this case the IOHMM represents the probabilistic relationship between the actions and the observations, with a hidden state variable. Although the HMMs such as those described in section 3 for speech recognition do represent the conditional probability of an observation sequence given a classification sequence, this is achieved by deterministically attaching a symbolic meaning to the different values of the state variable. Instead, in IOHMMs, the state variable is stochastically related to the output variable, and for classification problems, one can view the output sequence as the classification sequence and the input (observed) sequence as the conditioning sequence.

Potential advantages of IOHMMs over HMMs can be summarized as follows (and are described in more detail at the end of this section): the training criterion is discriminant, the emission and transition models can be naturally represented by flexible models such as ANNs, the problem of imbalance between transition and emission probabilities is reduced, and long-term dependencies may be more easily captured and learned.

In the next section we describe particular kinds of IOHMMs which have been proposed in the econometrics literature. In the following section, we present the EM algorithm which can be used for training both HMMs and synchronous HMMs.




next up previous
Next: Markov Switching Models Up: Markovian Models for Sequential Previous: Integrating Artificial Neural Networks

Yoshua Bengio
Tue Oct 7 08:34:36 EDT 1997