Standard Article

Partially Observable MDPs (POMDPS): Introduction and Examples

  1. Emine Yaylali,
  2. Julie S. Ivy

Published Online: 15 FEB 2011

DOI: 10.1002/9780470400531.eorms0646

Wiley Encyclopedia of Operations Research and Management Science

Wiley Encyclopedia of Operations Research and Management Science

How to Cite

Yaylali, E. and Ivy, J. S. 2011. Partially Observable MDPs (POMDPS): Introduction and Examples. Wiley Encyclopedia of Operations Research and Management Science. .

Author Information

  1. North Carolina State University, Edward P. Fitts Department of Industrial and Systems Engineering, Raleigh, North Carolina

Publication History

  1. Published Online: 15 FEB 2011

Abstract

A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process where the states of the model are not completely observable by the decision maker. Noisy observations provide a belief regarding the underlying state, while the decision maker has some control over the progression of the model through the selection of actions. In this article, we introduce POMDPs and discuss the relationship between Markov models and POMDPs. A general POMDP formulation and a wide range of POMDP applications from the literature are also presented.

Keywords:

  • MDP;
  • Decision making under uncertainty;
  • hidden Markov chain;
  • POMDP;
  • applications of POMDP