The Interplay Between Gesture and Speech in the Production of Referring Expressions: Investigating the Tradeoff Hypothesis
Article first published online: 2 MAR 2012
Copyright © 2012 Cognitive Science Society, Inc.
Topics in Cognitive Science
Volume 4, Issue 2, pages 232–248, April 2012
How to Cite
de Ruiter, J. P., Bangerter, A. and Dings, P. (2012), The Interplay Between Gesture and Speech in the Production of Referring Expressions: Investigating the Tradeoff Hypothesis. Topics in Cognitive Science, 4: 232–248. doi: 10.1111/j.1756-8765.2012.01183.x
- Issue published online: 11 APR 2012
- Article first published online: 2 MAR 2012
- Received 28 February 2010; received in revised form 30 August 2010; accepted 5 November 2010
- Iconic gestures;
- Referring expressions;
- Speech production;
- Gesture–speech tradeoff;
- Gesture–speech redundancy
The tradeoff hypothesis in the speech–gesture relationship claims that (a) when gesturing gets harder, speakers will rely relatively more on speech, and (b) when speaking gets harder, speakers will rely relatively more on gestures. We tested the second part of this hypothesis in an experimental collaborative referring paradigm where pairs of participants (directors and matchers) identified targets to each other from an array visible to both of them. We manipulated two factors known to affect the difficulty of speaking to assess their effects on the gesture rate per 100 words. The first factor, codability, is the ease with which targets can be described. The second factor, repetition, is whether the targets are old or new (having been already described once or twice). We also manipulated a third factor, mutual visibility, because it is known to affect the rate and type of gesture produced. None of the manipulations systematically affected the gesture rate. Our data are thus mostly inconsistent with the tradeoff hypothesis. However, the gesture rate was sensitive to concurrent features of referring expressions, suggesting that gesture parallels aspects of speech. We argue that the redundancy between speech and gesture is communicatively motivated.