Chapter 6. Discrete-Time Markov Decision Processes

  1. Henk C. Tijms

Published Online: 27 DEC 2004

DOI: 10.1002/047001363X.ch6

A First Course in Stochastic Models

A First Course in Stochastic Models

How to Cite

Tijms, H. C. (2003) Discrete-Time Markov Decision Processes, in A First Course in Stochastic Models, John Wiley & Sons, Ltd, Chichester, UK. doi: 10.1002/047001363X.ch6

Author Information

  1. Vrije Universiteit, Amsterdam, The Netherlands

Publication History

  1. Published Online: 27 DEC 2004
  2. Published Print: 14 MAR 2003

ISBN Information

Print ISBN: 9780471498803

Online ISBN: 9780470013632

SEARCH

Keywords:

  • Markov decision model;
  • policy-iteration algorithm;
  • linear programming and probabilistic constraints

Summary

This chapter contains sections titled:

  • Introduction

  • The Model

  • The Policy-improvement Idea

  • The Relative Value Function

  • Policy-iteration Algorithm

  • Linear Programming Approach

  • Value-iteration Algorithm

  • Convergence Proofs

  • Exercises

  • Bibliographic Notes

  • References