Sampling-based Planning with Approximate and Learned MDP Models
par/by Cosmin Paduraru
Computer Science Department
McGill University
Planning in Markov decision processes remains a challenging problem for artificial intelligence. Planning methods that sample transitions from a generative model of
the system's dynamics are appealing because they do not constrain the form of the model and they enable a flexible trade-off between planning and acting. A
limitation of existing methods for representing the generative model, however, is that they make restrictive assumptions about the structure of the problem or the
type of stochasticity in the world. In this talk I will present a new approach to sampling-based planning that does not make such strong assumptions, at the cost of
requiring the states to be represented as binary feature vectors. The BCD (binary cascade Dyna) architecture uses a cascade structure for sampling binary features;
the resulting generative model is used in a Dyna-style planning and learning ssystem. Empirical results in a continuous- state stochastic navigation problem and a
stochastic version of mountain car illustrates the advantages of BCD over model-free learning.