Standard Article

Semi-Markov Decision Processes

  1. Melike Baykal-Gürsoy

Published Online: 15 JUN 2010

DOI: 10.1002/9780470400531.eorms0757

Wiley Encyclopedia of Operations Research and Management Science

Wiley Encyclopedia of Operations Research and Management Science

How to Cite

Baykal-Gürsoy, M. 2010. Semi-Markov Decision Processes. Wiley Encyclopedia of Operations Research and Management Science. .

Author Information

  1. RUTCOR, CAIT, Rutgers University, Department of Industrial and Systems Engineering, New Brunswick, New Jersey

Publication History

  1. Published Online: 15 JUN 2010

Abstract

Considered are infinite horizon semi-Markov decision processes (SMDPs) with finite state and action spaces. Total expected discounted reward and long run average expected reward optimality criteria are reviewed. Solution methodology for each criterion is given; constraints and variance sensitivity are also discussed.

Keywords:

  • Markov decision process;
  • decision rule;
  • Weibull distribution;
  • unichain;
  • reward structure