6. Discounted Markov Decision Problems

  1. Martin L. Puterman

Published Online: 27 MAY 2008

DOI: 10.1002/9780470316887.ch6

Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming

How to Cite

Puterman, M. L. (1994) Discounted Markov Decision Problems, in Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, Inc., Hoboken, NJ, USA. doi: 10.1002/9780470316887.ch6

Author Information

  1. University of British Columbia

Publication History

  1. Published Online: 27 MAY 2008
  2. Published Print: 15 APR 1994

ISBN Information

Print ISBN: 9780471619772

Online ISBN: 9780470316887

SEARCH

Keywords:

  • Markov decision problems;
  • value iteration;
  • reward functions;
  • transition probabilities;
  • bounded rewards

Summary

The prelims comprise:

  • Policy Evaluation

  • Optimality Equations

  • Value Iteration and Its Variants

  • Policy Iteration

  • Modified Policy Iteration

  • Spans, Bounds, Stopping Criteria, and Relative Value Iteration

  • Action Elimination Procedures

  • Convergence of Policies, Turnpikes and Planning Horizons

  • Linear Programming

  • Countable-State Models

  • The Optimality of Structured Policies