Decision Making under Parameter Uncertainty

par/by Shie Mannor
Department of Electrical and Computer Engineering
McGill University

Markov decision processes are an effective tool in modeling decision-making in uncertain dynamic environments. The parameters of these models are often estimated from data, learned from experience or designed by hand. It is therefore not surprising that the actual performance of a chosen strategy often significantly differs from the designerĮs initial expectations due to unavoidable modeling ambiguity. In this talk, we consider two methodological approaches that enable the decision maker to take this uncertainty into account. The first approach we consider is a percentile optimization approach that allows the decision maker to naturally optimize a desired level of risk measured in terms of percentile performance. We show that some forms of this uncertainty can be efficiently solved and others are NP-hard. The second approach obtains the whole Pareto front of solutions that provide minimal cost subject to variable risk values. This allows the decision maker to weigh the potential gain by varying the level of risk that can be tolerated.