Definition and Examples of Continuous-Time Markov Chains
Published Online: 15 FEB 2011
Copyright © 2010 John Wiley & Sons, Inc. All rights reserved.
Wiley Encyclopedia of Operations Research and Management Science
How to Cite
Uzun, E. 2011. Definition and Examples of Continuous-Time Markov Chains. Wiley Encyclopedia of Operations Research and Management Science. .
- Published Online: 15 FEB 2011
This article consists of definitions and examples of continuous-time Markov chains (CTMCs). We first define a CTMC as a continuous-time stochastic process that has a countable state space and the Markov property. Then, we restrict our attention to time-homogenous CTMCs, for which we define the transition probability matrix. We further restrict our attention to regular CTMCs and provide an alternative definition of a CTMC. Next, we define the embedded discrete-time Markov chain and the generator matrix of a CTMC and investigate the possible ways of characterizing a CTMC. We complete the definitions by defining the transition rate diagram of a CTMC. Finally, we conclude the article by providing examples of CTMCs.
- markov chain;
- continuous-time stochastic process;
- poisson process;
- birth and death process