Learning and Using Task-Oriented Computed Vision Models of Human Behavior

Jesse Hoey

In this talk, I will discuss learning partially observable Markov decision process (POMDP) models of human interactive tasks, in which a decision-making agent must interact with a human to achieve its goals. Central to our work is the modeling of human behaviors within this context using computer vision. The functions we seek to learn are mappings from spatially and temporally extended observations (video sequences) to actions which are optimal with respect to some utility function (i.e. a policy). Given the large observation space of the video sequences, this task is one of learning spatio-temporal abstractions of the observations that can enable optimal action choices at the level of goals and utility. One of the most significant advantages of this type of learning is that it does not require labeled data from expert knowledge about which behaviors are significant in a particular interaction. Rather, the learning process discovers the significant behaviors, leading to a transportable system. A key idea is that only those behaviors which are "important" (in the sense that they enable or help the system to complete its goals) need to be distinguished perceptually. That is, we assume some observation subspace that spans the perceptual distinctions that need to be made in order to achieve goals. We find this subspace by learning dynamic Bayesian network models of human behaviors. The parameters of these models are learned from training data using an a-posteriori constrained optimization technique based on the expectation-maximization algorithm.

I will briefly overview POMDPs and multi-agent POMDPs, and expose some of the difficulties inherent in learning to act in a multi-agent situation. I will then describe the computer vision observation functions which we have experimented with. I will show how to learn the parameters of these models, and show how they can be applied to a gestural robot control experiment and a collaborative card game. Finally, I will describe my current work towards building these models for an assisted living task, and will discuss some of the outstanding issues.