Several important challenges for research on Markovian models remain:
What happens when we try to learn what the hidden state should represent? The state variable keeps some informations about the past sequence and discards others. It therefore captures the temporal dependencies. In [&make_named_href('', "node31.html#Bengio+Frasconi-jair95","[113]")], it was shown that, for Markovian models (including HMMs, IOHMMs, Markov switching models and Partially Observable Markov Decision Processes), learning of long-term dependencies in sequential data becomes exponentially more difficult as the span of these dependencies increases. However, it was found that this problem is not as bad for conditional models (such as IOHMMs, conditional Markov switching models and Partially Observable Markov Decision Processes) because the state to next-state transformation, being conditioned with extra information, is generally more deterministic.
One promising direction that was proposed to manage this problem is to split the state variable in multiple sub-state variables [&make_named_href('', "node31.html#Zoubin-nips8","[125]")], which may operate at different time scales [&make_named_href('', "node31.html#ElHihi+Bengio-nips8","[132]")], since the ``slow'' variables can more easily represent longer-term context.