DECISION STRATEGIES DETERMINED FROM THE SHAPE OF THE SAME–DIFFERENT ROC CURVE: WHAT ARE THE EFFECTS OF INCORRECT ASSUMPTIONS?
Article first published online: 26 NOV 2008
© 2008, The Author(s). Journal compilation © 2008, Wiley Periodicals, Inc.
Journal of Sensory Studies
Volume 23, Issue 6, pages 743–764, December 2008
How to Cite
HAUTUS, M. J., O'MAHONY, M. and LEE, H.-S. (2008), DECISION STRATEGIES DETERMINED FROM THE SHAPE OF THE SAME–DIFFERENT ROC CURVE: WHAT ARE THE EFFECTS OF INCORRECT ASSUMPTIONS?. Journal of Sensory Studies, 23: 743–764. doi: 10.1111/j.1745-459X.2008.00185.x
- Issue published online: 26 NOV 2008
- Article first published online: 26 NOV 2008
- Accepted for Publication November 7, 2007
The literature includes many studies that use the shape of the receiver operating characteristic (ROC) curve derived from the same–different test to ascertain the decision strategy used by the judge: a symmetrical ROC curve indicates a β -decision strategy, and an asymmetrical ROC curve indicates a τ -decision strategy. These studies have not considered the effect that breaches of the assumptions that underlie the standard same–different model would have on inferences drawn from the shape of the ROC curve. Three such assumptions are investigated: the equal-variance assumption; the assumption of no stimulus order effects; and the assumption that the discriminability of the two stimuli is the same when presented first or second in the test sequence. Simulations indicate that the shape of the ROC curve is very tolerant of the breach of these assumptions. Consequently, inferences about the decision strategy used by a judge are fairly robust, and confidence in published results is upheld.
The same–different test is frequently used to determine the decision strategy adopted by judges. There are several assumptions made about the same–different test that may, if they were breached, undermine this test's ability to indicate the judge's decision strategy. The simulations conducted in this research indicate that the shape of the same–different ROC curve is very robust to breaches of these assumptions. Consequently, food scientists can maintain their confidence in the inferences made in the research already published on this topic, and in their own future research and development of sensory testing methods.