Decision-making with Predictive State Representations (PSRs)

par/by Michal James

Toyota Research Labs

The ability to make decisions under uncertainty is a key component of building AI agents for complex dynamical systems. In this talk, I present decision-making algorithms for agents which model the environment using Predictive State Representations (PSRs) and an extension called memory-PSRs. PSRs and mPSRs require only the use of observable quantities in their representation, and so have the advantage of being grounded and verifiable in the agent's experience (as opposed to nominal-state based models such as POMDPs). The decision-making algorithms presented are extensions of POMDP algorithms which, in the case of mPSRs, take advantage of structure in the mPSR representation to improve performance.