Standard Article

Definition and Examples of Continuous-Time Markov Chains

  1. Evin Uzun

Published Online: 15 FEB 2011

DOI: 10.1002/9780470400531.eorms0236

Wiley Encyclopedia of Operations Research and Management Science

Wiley Encyclopedia of Operations Research and Management Science

How to Cite

Uzun, E. 2011. Definition and Examples of Continuous-Time Markov Chains. Wiley Encyclopedia of Operations Research and Management Science. .

Author Information

  1. University of North Carolina, Department of Statistics and Operations Research, Chapel Hill, North Carolina

Publication History

  1. Published Online: 15 FEB 2011

Abstract

This article consists of definitions and examples of continuous-time Markov chains (CTMCs). We first define a CTMC as a continuous-time stochastic process that has a countable state space and the Markov property. Then, we restrict our attention to time-homogenous CTMCs, for which we define the transition probability matrix. We further restrict our attention to regular CTMCs and provide an alternative definition of a CTMC. Next, we define the embedded discrete-time Markov chain and the generator matrix of a CTMC and investigate the possible ways of characterizing a CTMC. We complete the definitions by defining the transition rate diagram of a CTMC. Finally, we conclude the article by providing examples of CTMCs.

Keywords:

  • markov chain;
  • continuous-time stochastic process;
  • time-homogeneity;
  • regularity;
  • poisson process;
  • birth and death process