Definition and Examples of DTMCs
Published Online: 15 FEB 2011
Copyright © 2010 John Wiley & Sons, Inc. All rights reserved.
Wiley Encyclopedia of Operations Research and Management Science
How to Cite
Gautam, N. 2011. Definition and Examples of DTMCs. Wiley Encyclopedia of Operations Research and Management Science. .
- Published Online: 15 FEB 2011
The objective of this article is to describe a special type of stochastic process called discrete-time Markov chain (DTMC). We first provide definitions for DTMCs using simple language as well as mathematics. Then, we describe a systematic technique to model a system as a DTMC. Finally, we provide examples to both illustrate the modeling technique and motivate the breadth of real-world applications where DTMCs can be used.
- time homogeneous;
- transition probability matrix;
- stochastic process;
- Markov property