In many applications of HMMs such as speech recognition,
there are actually two sequences of interest: the
observation (e.g., acoustic) sequence,
, and the
classification (e.g. correct word)
sequence,
. The traditional approach to
HMM speech recognition is to consider independently
a different HMM for each word (or speech unit) and to learn the
distribution of acoustic subsequences associated with each
word or speech unit. By
concatenating the speech units associated
with the correct word sequence
, one can represent
the conditional acoustic probability
. An HMM that
is constrained by the knowledge of the correct word
sequence is called a constrained model, and
is illustrated in Figure 8.
On the other hand, during speech recognition, the
correct word sequence is unknown, and all the word
sequences allowed by the language model must be
taken into account. An HMM that allows all these
word sequences is called a recognition model
(and it is generally much larger than a constrained model).
It represents the joint distribution
(or, when summing over all possible state paths, the
observed sequence probability
).
Figure 8: Example of a constrained HMM, representing
the conditional distribution
.
Here
is an acoustic sequence and
is the sequence of two words /the/ and /dog/.
A maximum likelihood
criterion can then be applied to estimate the parameters
of the speech units that maximize the constrained
acoustic likelihood,
,
over all the training sequences (indexed by p above). For this purpose,
the EM or GEM algorithms described in section 5.2
are often used.
The above approach is called non-discriminant
because it is based on learning the acoustic distribution
associated with each speech unit (i.e., class-conditional
density functions),
rather than learning how the various linguistic classes
differ acoustically.
When trying to discriminate between
different interpretations
(i.e., different classes),
it is sufficient to know about these differences, e.g, using
or even directly describing the decision
surface in the space of acoustic sequences. The non-discriminant models
contain the additional information
.
Furthermore, they strongly rely on the assumptions
made on the form of the probability density of the
acoustic data
.
Since the models chosen to represent the data are
generally imperfect, it has been found that better
classification results can often be obtained when the objective
of learning is closer to the reduction of the number
of classification errors. Several approaches have been proposed
to train HMMs with a discriminant criterion.
The most common are the maximum a posteriori criterion,
to maximize
, and
the maximum mutual information
criterion [&make_named_href('',
"node31.html#BrownPhD","[67]"), &make_named_href('',
"node31.html#Bahl87","[68]"), &make_named_href('',
"node31.html#Nadas88","[69]")],
to maximize
.
The maximum mutual information criterion is therefore
obtained by comparing the log-probability of the constrained
model,
, with the log-probability
of the unconstrained recognition model,
allowing all the possible interpretations (word sequences).
Maximizing this criterion attempts to increase
the likelihood of the correct (i.e., constrained) model
while decreasing the likelihood of all the other models.
Other criteria have been proposed to approximate
the minimization of the number of classification
errors [&make_named_href('',
"node31.html#Juang92","[70]"), &make_named_href('',
"node31.html#Leprieur95","[71]")].
A gradient-based numerical optimization method is generally used with these criteria: the EM algorithm cannot be used in general (an exception is the synchronous or asynchronous Input-Output HMM with discrete observations, described in section 5).