## Introduction

A recent paper by Stephens *et al*. (2005) questions the rejection of null hypothesis testing (NHT) and calls for a ‘pluralism’ of analysis methods. Much of their paper restates prior criticisms of the common misuses of NHT, with which we are in agreement. The problems with NHT and *P*-values have been discussed extensively in the literature (for a list of > 400 citations see http://www.warnercnr.colostate.edu/~anderson/null.html (accessed 18 December 2006)), so we do not repeat them here. We agree with Stephens *et al*. (2005) that both NHT and information-theoretic (I-T) methods can be used inappropriately and careful attention must be paid to the use of all statistical methods. However, there are several occasions in Stephens *et al*. (2005) in which the potential for misunderstanding is great; thus we feel further clarity regarding I-T approaches is warranted. For example, we interpreted their call for plurality as the notion that combining IT methods and NHT in data analysis can provide stronger inference. The authors describe a study that combined usage of likelihood ratio testing and I-T methods as a powerful analysis approach. Further correspondence has indicated their intent was to suggest that I-T methods should be placed alongside NHT in the biologist's statistical toolbox and not to suggest that both approaches be used in concert with one another.

NHT remains in use in scientific research. NHT is not mathematically wrong; it is just relatively uninformative for scientific questions compared with modern analysis methods. Scientists are changing their views of inference as better methods, such as I-T and Bayesian, are replacing NHT. I-T and Bayesian methods provide statistical frameworks for pursuing the multiple-hypothesis approach to science advanced by Chamberlin (1897) and endorsed as strong inference by Platt (1964). They are also consistent with modern approaches to decision making in the face of competing ecological models, found in adaptive resource management (Walters 1986; Williams, Nichols & Conroy 2002). We do not object to NHT being in a statistical toolbox if used carefully. We would rarely use NHT ourselves but often we, as journal referees, would not argue against its use, for example as a means of assessing goodness-of-fit.

NHT and I-T represent two philosophically different views of data analysis and inference. NHT attempts to present a binary choice between the null hypothesis (H_{0}) and the alternative (H_{A}), based on an arbitrary α level and the resulting *P*-value, the probability of the data (*X*) and unobserved, more extreme, data given the null hypothesis, Prob(*X* | H_{0}). In contrast, I-T provides simple ways to quantify directly the evidence for two or more science hypotheses. This evidence usually stems from a simultaneous analysis of multiple hypotheses and includes a ranking of the models based on information loss, the probability of each model given the data [Prob(*H*_{j} |* X*) for *j*= 1, 2, … , *R*, where *R* is the number of models] and evidence ratios (Table 1).

Null hypothesis testing | Information-theoretic |
---|---|

P-values = Prob(X | H_{0}) | Ranking of H_{j} j= 1, …R |

Probability of hypothesis j= Prob(H_{j} | X) | |

Evidence ratios, hypothesis i vs. j | |

Model averaging | |

Unconditional estimates of precision |

Stephens *et al*.'s (2005) abbreviation ‘information-theoretic model comparison (ITMC)’ poorly denotes the breadth of this approach. Kullback–Leibler information and its asymptotic estimator (Akaike's information criterion) is much more than a ‘model comparator’. Hence we have adopted the abbreviation I-T in discussing this class of methods, including several procedures to allow rigorous inference from more than a single model (multimodel inference). Stephens *et al*. (2005) further state that NHT is a more appropriate tool for some questions that they later clarify as being single-parameter studies. We do not agree that NHT is more appropriate than I-T methods for the scenarios they presented, and offer an explanation for our disagreement. We wish to make two main points. First, we demonstrate that I-T is directly applicable in single-parameter problems and, moreover, that it is more informative than NHT. Secondly, we stress that developing scientific hypotheses is a difficult process and it should rightly be challenging. Hypothesizing is the cornerstone of science and must be given considerable thought. I-T methods encourage greater a priori thinking than NHT, which focuses on the testing of a null hypothesis against an alternative.