Performing high‐powered studies efficiently with sequential analyses
Abstract
Running studies with high statistical power, while effect size estimates in psychology are often inaccurate, leads to a practical challenge when designing an experiment. This challenge can be addressed by performing sequential analyses while the data collection is still in progress. At an interim analysis, data collection can be stopped whenever the results are convincing enough to conclude that an effect is present, more data can be collected, or the study can be terminated whenever it is extremely unlikely that the predicted effect will be observed if data collection would be continued. Such interim analyses can be performed while controlling the Type 1 error rate. Sequential analyses can greatly improve the efficiency with which data are collected. Additional flexibility is provided by adaptive designs where sample sizes are increased on the basis of the observed effect size. The need for pre‐registration, ways to prevent experimenter bias, and a comparison between Bayesian approaches and null‐hypothesis significance testing (NHST) are discussed. Sequential analyses, which are widely used in large‐scale medical trials, provide an efficient way to perform high‐powered informative experiments. I hope this introduction will provide a practical primer that allows researchers to incorporate sequential analyses in their research. Copyright © 2014 John Wiley & Sons, Ltd.
Number of times cited: 27
- Daniel S. Quintana, Revisiting non-significant effects of intranasal oxytocin using equivalence testing, Psychoneuroendocrinology, 10.1016/j.psyneuen.2017.10.010, 87, (127-130), (2018).
- Casper Albers and Daniël Lakens, When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias, Journal of Experimental Social Psychology, 74, (187), (2018).
- John K. Sakaluk and Cynthia A. Graham, Promoting Transparent Reporting of Conflicts of Interests and Statistical Analyses at The Journal of Sex Research, The Journal of Sex Research, 55, 1, (1), (2018).
- Leif D. Nelson, Joseph Simmons and Uri Simonsohn, Psychology's Renaissance, Annual Review of Psychology, 69, 1, (511), (2018).
- Barbara A. Spellman, Elizabeth A. Gilbert and Katherine S. Corker, Open Science, Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience, (1-47), (2018).
- Arianne E. Eason, J. Kiley Hamlin and Jessica A. Sommerville, A Survey of Common Practices in Infancy Research: Description of Policies, Consistency Across and Within Labs, and Suggestions for Improvements, Infancy, 22, 4, (470-491), (2017).
- Lisa M. Oakes, Sample Size, Statistical Power, and False Conclusions in Infant Looking‐Time Research, Infancy, 22, 4, (436-469), (2017).
- Courtland S. Hyatt, Chelsea E. Sleep, W. Keith Campbell and Joshua D. Miller, Exposure to Celebrities as a Possible Explanatory Mechanism in the Perception of American Narcissism, Collabra: Psychology, 3, 1, (4), (2017).
- Martin S. Hagger, Daniel F. Gucciardi and Nikos L. D. Chatzisarantis, On Nomological Validity and Auxiliary Assumptions: The Importance of Simultaneously Testing Effects in Social Cognitive Theories Applied to Health Behavior and Some Guidelines, Frontiers in Psychology, 8, (2017).
- , Simulating Social Complexity, (229)
- John K. Kruschke and Torrin M. Liddell, The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective, Psychonomic Bulletin & Review, (2017).
- Daniël Lakens, Equivalence Tests, Social Psychological and Personality Science, 8, 4, (355), (2017).
- Samantha F. Anderson, Ken Kelley and Scott E. Maxwell, Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty, Psychological Science, 28, 11, (1547), (2017).
- Federica Durante, Susan T. Fiske, Michele J. Gelfand, Franca Crippa, Chiara Suttora, Amelia Stillwell, Frank Asbrock, Zeynep Aycan, Hege H. Bye, Rickard Carlsson, Fredrik Björklund, Munqith Dagher, Armando Geller, Christian Albrekt Larsen, Abdel-Hamid Abdel Latif, Tuuli Anna Mähönen, Inga Jasinskaja-Lahti and Ali Teymoori, Ambivalent stereotypes link to peace, conflict, and inequality across 38 nations, Proceedings of the National Academy of Sciences, 114, 4, (669), (2017).
- Konrad Neumann, Ulrike Grittner, Sophie K. Piper, Andre Rex, Oscar Florez-Vargas, George Karystianis, Alice Schneider, Ian Wellwood, Bob Siegerink, John P. A. Ioannidis, Jonathan Kimmelman and Ulrich Dirnagl, Increasing efficiency of preclinical research by group sequential designs, PLOS Biology, 15, 3, (e2001307), (2017).
- Ryan Dwyer, Kostadin Kushlev and Elizabeth Dunn, Smartphone use undermines enjoyment of face-to-face social interactions, Journal of Experimental Social Psychology, (2017).
- Sofia Calderon, Erik Mac Giolla, Pär Anders Granhag and Karl Ask, Do True and False Intentions Differ in Level of Abstraction? A Test of Construal Level Theory in Deception Contexts, Frontiers in Psychology, 8, (2017). Companion Publication of the 21st International Conference on Intelligent User Interfaces - IUI '16 Companion, (2016).Bart P. Knijnenburg610.1145/2876456.2882846
- Jeff Miller and Rolf Ulrich, Optimizing Research Payoff, Perspectives on Psychological Science, 11, 5, (664), (2016).
- Andrew H. Hales, Does the conclusion follow from the evidence? Recommendations for improving research, Journal of Experimental Social Psychology, 66, (39), (2016).
- Roger Giner-Sorolla, Approaching a fair deal for significance and other concerns, Journal of Experimental Social Psychology, 65, (1), (2016).
- Anna Elisabeth van 't Veer and Roger Giner-Sorolla, Pre-registration in social psychology—A discussion and suggested template, Journal of Experimental Social Psychology, 67, (2), (2016).
- Richard Kunert, Internal conceptual replications do not increase independent replication success, Psychonomic Bulletin & Review, 23, 5, (1631), (2016).
- Patrizio E. Tressoldi and David Giofré, The pervasive avoidance of prospective statistical power: major consequences and practical solutions, Frontiers in Psychology, 6, (2015).
- Daniël Lakens, On the challenges of drawing conclusions fromp-values just below 0.05, PeerJ, 3, (e1142), (2015).
- Chao Zhang, Daniël Lakens and Wijnand A. IJsselsteijn, The illusion of nonmediation in telecommunication: Voice intensity biases distance judgments to a communication partner, Acta Psychologica, 157, (101), (2015).
- Kai Sassenberg, Dominique Muller and Karl Christoph Klauer, Methods and statistics in social psychology—refinements and new developments, European Journal of Social Psychology, 44, 7, (671-672), (2014).
