Permutation methods

Authors

  • Kenneth J. Berry,

    Corresponding author
    1. Department of Sociology, Colorado State University, Fort Collins, CO, USA
    • Department of Sociology, Colorado State University, Fort Collins, CO, USA
    Search for more papers by this author
  • Janis E. Johnston,

    1. United States Department of Agriculture, Food and Nutrition Service, Alexandria, VA, USA
    Search for more papers by this author
    • The views expressed in this article are those of the author and do not necessarily reflect the position or policy of the United States Department of Agriculture or the United States government.

  • Paul W. Mielke Jr

    1. Department of Statistics, Colorado State University, Fort Collins, CO, USA
    Search for more papers by this author

Abstract

Permutation tests are a paradox of old and new. Permutation tests pre-date most traditional parametric statistics, but only recently have become part of the mainstream discussion regarding statistical testing. Permutation tests follow a permutation or ‘conditional on errors’ model whereby a test statistic is computed on the observed data, then (1) the data are permuted over all possible arrangements of the data—an exact permutation test; (2) the data are used to calculate the exact moments of the permutation distribution—a moment approximation permutation test; or (3) the data are permuted over a subset of all possible arrangements of the data—a resampling approximation permutation test. The earliest permutation tests date from the 1920s, but it was not until the advent of modern day computing that permutation tests became a practical alternative to parametric statistical tests. In recent years, permutation analogs of existing statistical tests have been developed. These permutation tests provide noteworthy advantages over their parametric counterparts for small samples and populations, or when distributional assumptions cannot be met. Unique permutation tests have also been developed that allow for the use of Euclidean distance rather than the squared Euclidean distance that is typically employed in parametric tests. This overview provides a chronology of the development of permutation tests accompanied by a discussion of the advances in computing that made permutation tests feasible. Attention is paid to the important differences between ‘population models’ and ‘permutation models’, and between tests based on Euclidean and squared Euclidean distances. WIREs Comp Stat 2011 3 527–542 DOI: 10.1002/wics.177

For further resources related to this article, please visit the WIREs website.

Ancillary