Get access

Statistical properties of population differentiation estimators under stepwise mutation in a finite island model


  • F. Balloux,

    1. University of Bern, CH-3032 Hinterkappelen-Bern, Switzerland,
    2. University of Edinburgh, Institute of Cell, Animal and Population Biology, King’s Buildings, West Mains Road, Edinburgh EH9 3JT, UK
    Search for more papers by this author
  • J. Goudet

    Corresponding author
    1. Institute of Ecology, Biology Building, University of Lausanne, CH-1015 Lausanne, Switzerland,
    Search for more papers by this author

Jérôme Goudet. Fax: + 41 21 692 41 05; E-mail:


Microsatellite loci mutate at an extremely high rate and are generally thought to evolve through a stepwise mutation model. Several differentiation statistics taking into account the particular mutation scheme of the microsatellite have been proposed. The most commonly used is inline image, which is independent of the mutation rate under a generalized stepwise mutation model. inline image and inline image are commonly reported in the literature, but often differ widely. Here we compare their statistical performances using individual-based simulations of a finite island model. The simulations were run under different levels of gene flow, mutation rates, population number and sizes. In addition to the per locus statistical properties, we compare two ways of combining inline image over loci. Our simulations show that even under a strict stepwise mutation model, no statistic is best overall. All estimators suffer to different extents from large bias and variance. While inline image better reflects population differentiation in populations characterized by very low gene-exchange, inline image gives better estimates in cases of high levels of gene flow. The number of loci sampled (12, 24, or 96) has only a minor effect on the relative performance of the estimators under study. For all estimators there is a striking effect of the number of samples, with the differentiation estimates showing very odd distributions for two samples.