Towards Competitive Instead of Biased Testing of Heuristics: A Reply to Hilbig and Richter (2011)
Version of Record online: 10 JAN 2011
Copyright © 2011 Cognitive Science Society, Inc.
Topics in Cognitive Science
Volume 3, Issue 1, pages 197–205, January 2011
How to Cite
Brighton, H. and Gigerenzer, G. (2011), Towards Competitive Instead of Biased Testing of Heuristics: A Reply to Hilbig and Richter (2011). Topics in Cognitive Science, 3: 197–205. doi: 10.1111/j.1756-8765.2010.01124.x
- Issue online: 10 JAN 2011
- Version of Record online: 10 JAN 2011
- Received 8 September 2010; received in revised form 4 November 2010; accepted 4 November 2010
- Simple heuristics;
- Recognition heuristic;
- Homo heuristicus;
- Biased testing
Our programmatic article on Homo heuristicus (Gigerenzer & Brighton, 2009) included a methodological section specifying three minimum criteria for testing heuristics: competitive tests, individual-level tests, and tests of adaptive selection of heuristics. Using Richter and Späth’s (2006) study on the recognition heuristic, we illustrated how violations of these criteria can lead to unsupported conclusions. In their comment, Hilbig and Richter conduct a reanalysis, but again without competitive testing. They neither test nor specify the compensatory model of inference they argue for. Instead, they test whether participants use the recognition heuristic in an unrealistic 100% (or 96%) of cases, report that only some people exhibit this level of consistency, and conclude that most people would follow a compensatory strategy. We know of no model of judgment that predicts 96% correctly. The curious methodological practice of adopting an unrealistic measure of success to argue against a competing model, and to interpret such a finding as a triumph for a preferred but unspecified model, can only hinder progress. Marewski, Gaissmaier, Schooler, Goldstein, and Gigerenzer (2010), in contrast, specified five compensatory models, compared them with the recognition heuristic, and found that the recognition heuristic predicted inferences most accurately.