10. Markov Models

  1. Nicky J. Welton1,
  2. Alexander J. Sutton2,
  3. Nicola J. Cooper2,
  4. Keith R. Abrams2 and
  5. A.E. Ades1

Published Online: 3 APR 2012

DOI: 10.1002/9781119942986.ch10

Evidence Synthesis for Decision Making in Healthcare

Evidence Synthesis for Decision Making in Healthcare

How to Cite

Welton, N. J., Sutton, A. J., Cooper, N. J., Abrams, K. R. and Ades, A.E. (2012) Markov Models, in Evidence Synthesis for Decision Making in Healthcare, John Wiley & Sons, Ltd, Chichester, UK. doi: 10.1002/9781119942986.ch10

Author Information

  1. 1

    School of Social and Community Medicine, University of Bristol, UK

  2. 2

    Department of Health Sciences, University of Leicester, UK

Publication History

  1. Published Online: 3 APR 2012
  2. Published Print: 11 MAY 2012

ISBN Information

Print ISBN: 9780470061091

Online ISBN: 9781119942986



  • Binomial likelihood;
  • continuous time model;
  • decision analysis;
  • Dirichlet distribution;
  • discrete time model;
  • Markov models;
  • Markov transition parameters;
  • multinomial likelihood;
  • prophylactic treatment;
  • transition probability


This chapter focuses on Markov models, which can be used to describe transitions between states where events can repeat over time. It begins by describing the Markov models, how they can be viewed in both continuous and discrete time decision model, and how to build a decision analysis around a Markov model. The chapter then illustrates, using an example data set on prophylactic treatment for asthma in children, how the state transition parameters can be estimated from a single trial and the results integrated in a decision model. Finally, it discusses how transition parameters can be estimated from a synthesis of available evidence, in particular focusing on different formats for reporting results that may have been used in different studies. Transition parameters may be estimated jointly from aggregated data from a single study using a multinomial likelihood and a Dirichlet prior.

Controlled Vocabulary Terms

binomial distribution; Dirichlet distribution; likelihood; Monte Carlo Markov Chain; multinomial distribution; Poisson distribution; transition probability