• Allele frequency;
  • Bayesian modeling;
  • Beta mixture;
  • FST;
  • Inbreeding coefficient;
  • Reversible jump algorithm

Summary An important fraction of recently generated molecular data is dominant markers. They contain substantial information about genetic variation but dominance makes it impossible to apply standard techniques to calculate measures of genetic differentiation, such as F-statistics. In this article, we propose a new Bayesian beta-mixture model that more accurately describes the genetic structure from dominant markers and estimates multiple FSTs from the sample. The model also has important application for codominant markers and single-nucleotide polymorphism (SNP) data. The number of FST is assumed unknown beforehand and follows a random distribution. The reversible jump algorithm is used to estimate the unknown number of multiple FSTs. We evaluate the performance of three split proposals and the overall performance of the proposed model based on simulated dominant marker data. The model could reliably identify and estimate a spectrum of degrees of genetic differentiation present in multiple loci. The estimates of FSTs also incorporate uncertainty about the magnitude of within-population inbreeding coefficient. We illustrate the method with two examples, one using dominant marker data from a rare orchid and the other using codominant marker data from human populations.