Approximate Dynamic Programming I: Modeling
Published Online: 14 JAN 2011
Copyright © 2010 John Wiley & Sons, Inc. All rights reserved.
Wiley Encyclopedia of Operations Research and Management Science
How to Cite
Powell, W. B. 2011. Approximate Dynamic Programming I: Modeling. Wiley Encyclopedia of Operations Research and Management Science. .
- Published Online: 14 JAN 2011
The first step in solving a stochastic optimization problem is providing a mathematical model. How the problem is modeled can impact the solution strategy. In this article, we provide a flexible modeling framework that uses a classic control-theoretic framework, avoiding devices such as one-step transition matrices. We describe the five fundamental elements of any stochastic, dynamic program. Different notational conventions are introduced, and the types of policies that can be used to guide decisions are described in detail. This discussion puts approximate dynamic programming in the context of a variety of other algorithmic strategies by using the modeling framework to describe a wide range of policies. A brief discussion of model-free programming is also provided.
- approximate dynamic programming;
- Markov decision process;
- state variable;
- transition function;
- model-free dynamic programming