P value functions: An underused method to present research results and to promote quantitative reasoning

Abstract

Null hypothesis significance testing has received a great amount of attention in recent years in the light of the reproducibility crisis of science. Recently, there have been calls to retire the dichotomization of study results into “significant” or “not significant” depending on whether the P value crosses some threshold or not. Ways of improving the interpretation of P values and confidence intervals are therefore needed. We illustrate the use of P value functions, which display confidence limits and P values for any confidence level for a parameter. P value functions accessibly display a wealth of information: point estimate for the parameter, one‐sided and two‐sided confidence limits at any level, and one‐sided and two‐sided P values for any null and non‐null value and the counternull value. Presenting several recent examples from the literature, we show how P value functions can be applied to present evidence and to make informed statistical inferences without resorting to dichotomization. We argue that P value functions are more informative than commonly used summaries of study results such as single P values or confidence intervals. P value functions require minimal retraining, are easily interpreted, and show potential to fix many of the common misinterpretation of P values and confidence intervals. To facilitate the adoption of P value functions, we released an R package for creating P value functions for several commonly used estimates.