risk-sensitive planning

(archived 2017)

Many real-world planning problems occur in environments where there may be incomplete information or where actions may not always lead to the same results. Examples include planning for retirement, where the state of the economy in the future is uncertain, and planning in logistics, where the duration of travel between two cities is uncertain due to potential congestion.

A Markov decision process (MDP) is a popular framework for modeling decision making in these kinds of problems, where an agent needs to plan a sequence of actions that maximizes its chances of reaching its goal. A partially observable MDP (POMDP) is an extension where the world that the agent is operating in is only partially observable, and a decentralized (PO)MDP is an extension where a team of agents needs to collectively plan their joint actions.

We proposed the risk-sensitive MDP and POMDP models, where the objective is to find a policy that maximizes the probability of reaching a goal state. Such a model and its accompanying algorithms are useful in high-risk applications, where a single failure can be catastrophic.

sponsors

CAREER: Decentralized Constraint-based Optimization for Multi-Agent Planning and Coordination.
National Science Foundation (2016 – 2021).


selected publications

                1. AAAI
                  Solving Risk-Sensitive POMDPs With and Without Cost Observations
                  Ping Hou, William Yeoh, and Pradeep Varakantham
                  In AAAI Conference on Artificial Intelligence, 2016
                  1. ICAPS
                    Revisiting Risk-Sensitive MDPs: New Algorithms and Results
                    Ping Hou, William Yeoh, and Pradeep Varakantham
                    In International Conference on Automated Planning and Scheduling, 2014