Standard Article

Asymptotic Behavior of Discrete-Time Markov Chains

  1. Eylem Tekin

Published Online: 14 JAN 2011

DOI: 10.1002/9780470400531.eorms0064

Wiley Encyclopedia of Operations Research and Management Science

Wiley Encyclopedia of Operations Research and Management Science

How to Cite

Tekin, E. 2011. Asymptotic Behavior of Discrete-Time Markov Chains. Wiley Encyclopedia of Operations Research and Management Science. .

Author Information

  1. Texas A&M University, Department of Industrial and Systems Engineering, College Station, Texas

Publication History

  1. Published Online: 14 JAN 2011

Abstract

This article summarizes the theoretical foundations for analyzing the asymptotic behavior of discrete-time Markov chains (DTMC). Consider a system that evolves randomly over time. Let Xn denote the state of the system at time n = 0, 1, … . If, for all n ≥ 0, the future probability distribution of the system depends only on the current state and is independent of the past, the stochastic process {Xn, n ≥ 0} is called a DTMC. In this article, we study the distribution of Xn as n [RIGHTWARDS ARROW] ∞. We first introduce several concepts for classifying the states of a DTMC. These concepts include irreducibility, periodicity, recurrence, and transience. Using these concepts, we next describe the necessary and sufficient conditions for the existence and uniqueness of a limiting distribution, and explain how to compute a limiting distribution, if it exists.

Keywords:

  • Markov chains;
  • asymtotic behavior;
  • limiting distribution;
  • irreducibility;
  • periodicity;
  • recurrence;
  • transience