Standard Article

Definition and Examples of DTMCs

  1. Natarajan Gautam

Published Online: 15 FEB 2011

DOI: 10.1002/9780470400531.eorms0237

Wiley Encyclopedia of Operations Research and Management Science

Wiley Encyclopedia of Operations Research and Management Science

How to Cite

Gautam, N. 2011. Definition and Examples of DTMCs. Wiley Encyclopedia of Operations Research and Management Science. .

Author Information

  1. Texas A&M University, Department of Industrial and Systems Engineering, College Station, Texas

Publication History

  1. Published Online: 15 FEB 2011

Abstract

The objective of this article is to describe a special type of stochastic process called discrete-time Markov chain (DTMC). We first provide definitions for DTMCs using simple language as well as mathematics. Then, we describe a systematic technique to model a system as a DTMC. Finally, we provide examples to both illustrate the modeling technique and motivate the breadth of real-world applications where DTMCs can be used.

Keywords:

  • DTMC;
  • time homogeneous;
  • transition probability matrix;
  • stochastic process;
  • Markov property