In Praise of a Model but Not Its Conclusions: Commentary on Cooper, Catmur, and Heyes (2012)

Authors


Correspondence should be sent to Bennett I. Bertenthal, 1101 E. Tenth St., Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, 47405. E-mail: bbertent@indiana.edu

Abstract

Cooper et al. (this issue) develop an interactive activation model of spatial and imitative compatibilities that simulates the key results from Catmur and Heyes (2011) and thus conclude that both compatibilities are mediated by the same processes since their single model can predict all the results. Although the model is impressive, the conclusions are premature because they are based on an incomplete review of the relevant literature and because the model includes some questionable assumptions. Moreover, a competing model (Scheutz & Bertenthal, 2012) is introduced that suggests the two compatibilities are not mediated by the same processes. We propose that more research is necessary before concluding that spatial and imitative compatibilities are mediated by the same processes.

1. Introduction

Cooper et al. (in press; henceforth referred to as CCH) address a fundamental question concerning automatic imitation, namely whether it is a function of a domain general process involving associative learning or instead a function of a more specialized process. Their interactive activation model of spatial and imitative compatibilities simulates the key results from two experiments revealing the independent effects of these compatibilities (Catmur & Heyes, 2011), and based on the success of this model, the authors conclude that both compatibilities depend on the same underlying processes. While the model and the simulations are impressive, the conclusion is premature. In this commentary, we discuss three reasons for this view: (a) The reviewed literature is very selective and the conclusions are over-generalized; (b) some of the key assumptions of the model are questionable; and (c) competing computational models challenge the results presented by CCH, suggesting instead that spatial and imitative compatibilities are not mediated by the same processes.

2. Selective review of the literature

One of the motivations for the model developed by CCH was to support the conclusion by Catmur and Heyes (2011) that the same processes could explain both spatial and imitative compatibilities. According to these authors, their results contradicted the view expressed by Bertenthal, Longo, and Kosobud (2006) that time course differences in spatial and imitative compatibilities are reflective of different processes. Catmur and Heyes (2011) dismissed this interpretation by claiming that the results were an artifact of the design, but their evidence was based on a questionable criticism of the results from Bertenthal et al. (2006); moreover, the evidence for concluding that the same processes was sufficient for explaining both compatibilities ignored results from relevant follow-up studies (Boyer, Longo, & Bertenthal, 2012; Longo & Bertenthal, 2009; Longo, Kosobud, & Bertenthal, 2008).

We begin by reviewing the critical results of Experiment 3 in Bertenthal et al. (2006) because these results were not confounded by strategic factors as suggested by CCH, and furthermore, because it is important to set the record straight as this misrepresentation has been now repeated in two separate reports (Catmur & Heyes, 2011; CCH) and “sheer repetition sometimes gains a certain respect and momentum unrelated to merit” (Roberts & Pashler, 2000, p. 366). In the original critique by Catmur and Heyes (2011), they speculated that presenting compatible and incompatible trials in separate blocks enabled participants to develop different strategies for responding to the spatial and imitative compatibility tasks, and it is the development of these strategies (and not the underlying processes responsible for spatial and imitative compatibilities) that are responsible for the observed time course differences. Although this interpretation is highly speculative, they suggest that it is supported by an additional analysis of their own findings comparing response times across the four consecutive blocks of trials administered to participants. According to Catmur and Heyes (2011), this analysis should reveal an interaction between block and compatibility modality if Bertenthal et al. (2006) were correct about different time courses for spatial and imitative compatibilities. This interaction was not observed, however, and Catmur and Heyes (2011) thus conclude that there is no evidence for dissociable processes to explain spatial and imitative compatibilities.

The problem with the preceding critique is that Bertenthal et al. (2006) never reported time course differences for spatial and imitative compatibilities across the entire experiment, but rather time course differences within blocks of 20 trials. These time course differences were transitory and were repeated in each successive block because they were specifically triggered by the task switching that occurred at the beginning of each block. Thus, the analysis by Catmur and Heyes (2011) showing no interaction between the compatibility effects across the entire experiment does not in any way refute the specific time course differences reported by Bertenthal et al. (2006). Moreover, these time course differences could not be attributed simply to strategic differences, because both spatial and imitative compatibilities reflect automatic and involuntary processing and thus are mostly immune to strategic processing (e.g., Shiffrin & Schneider, 1977). Contrary to the suggestion by Catmur and Heyes (2011) and CCH that blocking trials as a function of compatibility or incompatibility resulted in different strategies that could explain the changing compatibility effects, we contend that the time course differences were a function of the differential strength and stability of the two compatibility effects. Although the evidence from this study was insufficient to definitively support the claim of dissociable processes for explaining spatial and imitative compatibilities, it was certainly not confounded in the manner suggested by CCH.

Converging evidence for this conclusion was provided in follow-up studies by Bertenthal and colleagues. In these more recent studies, the experimental paradigm was identical to that used by Bertenthal et al. (2006); participants were presented with a tapping index or middle finger on the left or right hand and were instructed to respond to either the spatial cue (left vs. right) or to the imitative cue (index vs. middle finger) by pressing a key with the index or middle finger of their right hand. When responding to the tapping finger on the left hand, the spatial cue was imitatively compatible with the responding finger and the imitative cue was spatially compatible with the responding finger. When responding to the tapping finger on the right hand, the responses were either spatially or imitatively incompatible with the stimulus cues (see Fig. 1, left panels). Response times are typically faster to compatible than incompatible stimuli. Yet the results from these follow-up studies revealed that imitative compatibility effects were attenuated or eliminated by perturbing the naturalness of the stimulus, whereas spatial compatibility effects were not (Longo & Bertenthal, 2009; Longo et al., 2008). For example, the imitative compatibility effect of observing a task-irrelevant moving finger was eliminated when the finger appeared to move in a biomechanically impossible manner, whereas the facilitating effect of spatial compatibility was not affected (Longo et al., 2008). These results suggest that imitative compatibility is modulated by the human-like similarity between the featural and kinematic properties of the stimulus and the response, whereas this similarity does not appear relevant to spatial compatibility.

Figure 1.

The four panels depict the relation between the stimulus and response for compatible and incompatible trials in the stimulus response (S-R) and opposite stimulus response (OS-R) compatibility tasks. Solid lines connecting the stimulus and response fingers depict the S-R mapping specified by the imperative stimulus. Dashed lines connecting the stimulus and response fingers depict the automatic S-R mapping of the stimulus. The two top panels correspond to the spatial compatibility condition: The task is to imitate the tapping finger (e.g., index finger) with the same finger of the right hand and the task-irrelevant stimulus is the left-right position of the fingers. In the left panel (S-R Task), the left-hand stimulus corresponds to the compatible condition—participants respond to the tapping of the left index finger with their index finger (i.e., stimulus and response are spatially congruent). The right-hand stimulus corresponds to the incompatible condition (i.e., stimulus and response are spatially incongruent). In the right panel (OS-R Task), the right-hand stimulus corresponds to the spatially compatible condition and the left-hand stimulus corresponds to the spatially incompatible condition. The two lower panels correspond to the imitative compatibility condition: The task is to respond to the left or right tapping finger with the corresponding left or right index or middle finger, respectively, and the task-irrelevant stimulus is the anatomical identity of the fingers. In the left panel (S-R Task), the left-hand stimulus corresponds to the compatible condition—participants respond to the tapping of the left index finger with their left index finger (i.e., stimulus and response correspond to the same anatomical finger). The right-hand stimulus corresponds to the incompatible condition—participants respond to the tapping index finger with their middle finger (i.e., stimulus and response correspond to different fingers). In the right panel (OS-R Task), the right-hand stimulus corresponds to the compatible condition and the left-hand stimulus corresponds to the incompatible condition (adapted from Boyer et al., 2012).

This evidence for dissociation between spatial and imitative compatibilities is not limited to our lab. A number of recent studies reveal that imitative compatibility is modulated as well by the inferred goals and intentions associated with the agent executing the movements (Liepelt, Von Cramon, & Brass, 2008; Liepelt & Brass, 2010; Wang, Newport, & Hamilton, 2011). It is difficult to imagine how this sort of mental state attribution would affect spatial compatibility, especially since this process is not restricted to human movements. One interpretation for the imitative compatibility findings is that the perception of others' movements is mediated by a bi-directional process involving bottom-up processing of sensory information as well as top-down processing of mental states (Teufel, Fletcher, & Davis, 2010). These authors emphasize that all socially relevant information is not directly perceived, but rather depends on the observers' beliefs about the intentionality and animacy of the observed movements. Thus, this interactive process should modulate automatic imitation, but there is no obvious reason why it would modulate the effects of spatial compatibility.

At the very least, the preceding results suggest that automatic imitation is not always operationalized in the manner presented by CCH. In fact, the same applies to spatial compatibility which can be modulated by both attentional and intentional factors (Hommel, 1993). The difference between the two compatibilities, however, is that imitative compatibility, but not spatial compatibility, is modulated by mental state as well as the surface appearance and kinematic properties of the stimulus (Longo & Bertenthal, 2009; Teufel et al., 2010). Moreover, “automatic imitation in the wild,” human mimicry, reveals that mimicry is a subtle and flexible behavior sensitive to social context, the people involved, and the specific goals of the interaction (Chartrand & van Baaren, 2009; Wang & Hamilton, 2012). In contrast, the model proposed by CCH assumes that automatic imitation is determined primarily by prior sensorimotor experience and is modulated only by the properties of the stimulus. Based on this assumption and the results from testing their model, CCH conclude that spatial and imitative compatibility are mediated by the same processes. Is this conclusion justified given that it is based on only one experimental paradigm assessing automatic imitation and spatial compatibility in one specific context? If the evidence is restricted to only those studies reviewed by CCH, then the above conclusion is plausible, but as this brief review suggests there is additional evidence that should be considered and cause CCH to be more circumspect in their conclusions.

3. Insufficient evidence for key assumptions of model

The second reason for questioning the conclusions of this study concerns three assumptions of the model that are foundational to its architecture: (a) Spatial compatibility effects are generally stronger than imitative compatibility effects, (b) excitation of task-irrelevant stimulus nodes is transitory while excitation of task-relevant stimulus nodes is sustained, and (c) temporal onset of excitation for movement location occurs earlier than for finger identification. These assumptions are critical to the simulation of the response times, but they lack firm empirical support because the data are either unavailable or inconsistent. For example, it is true that Catmur and Heyes (2011) and Bertenthal et al. (2006) reported stronger spatial than imitative compatibility effects, but Brass, Bekkering, and Prinz (2001) reported the opposite result. Moreover, the designs of the studies by Catmur and Heyes (2011) and Bertenthal et al. (2006) cannot directly establish whether the compatibility effects are a function of facilitation by the task-irrelevant stimulus, interference and inhibition by the task-irrelevant stimulus, or both. In contrast, Brass et al. (2001) include a baseline condition revealing that the compatibility effect is a function of both facilitation as well as interference.

With regard to whether task-irrelevant stimulation is transitory, the evidence is again unclear because the RT distributions for both spatial and imitative compatibilities reported by Boyer et al. (2012) suggest that there is no significant decrease in these effects even at the longest response times. Although the compatibility effect typically decreases with increasing response times for the standard left-right Simon effect (Simon, Acosta, Mewaldt, & Speidel, 1976), it remains stable or increases across the RT distributions for more complex tasks, such as object-based or word-based Simon effects (Proctor, Miles, & Baroni, 2011). According to Proctor et al. (2011), our understanding of the Simon effect across RT distributions is incomplete, and it is therefore difficult to predict when and why the compatibility effect will increase or decrease. Thus, it is simply not clear from the data presented by CCH or by Boyer et al. (2012) whether the stimulus-irrelevant activation consistently decays before a response is selected.

The third assumption concerning the differential onsets for activation of spatial and imitative compatibilities is presumably based on theory rather than empirical evidence. Curiously, there is no theory presented by CCH or by Catmur and Heyes (2011), who merely speculate that the identification of the finger will take longer to process than the relative spatial location of the finger. It would have been helpful for the authors to make their theory more explicit, but in the absence of a detailed explanation based on explicit psychological principles, the reasons seem elusive. Moreover, Boyer et al. (2012) present evidence that the response times for the identification of a task-relevant finger stimulus (index vs. middle) or its relative spatial location (left vs. right) are not significantly different. Although this result relates to the identification of the task-relevant stimulus rather than the task-irrelevant stimulus, there is no logical reason why this difference should affect the speed of identification. Thus, the evidence relating to this assumption that the speed of identifying finger location and spatial identity are different is neither supported by theory or data.

These inconsistencies between the empirical evidence and the assumptions of the model create a conundrum. On the one hand, the assumptions seem correct because they result in a model that successfully simulates the results. On the other hand, these assumptions lack independent validation. In the absence of more definitive evidence, they seem to prematurely constrain the family of models that were tested. As such, it is impossible to know whether any other models would fit the data as well or perhaps even better.

4. Competing data and models

A critical question not considered by CCH is whether the stimulus response compatibility data that they simulate with their model is sufficiently varied to differentiate between models. In essence, CCH claim that the success of the same model to simulate both spatial and imitative compatibility effects implies that the two compatibilities are mediated by the same processes. The problem with this conclusion is that both spatial and imitative compatibilities show the same pattern of responding in a stimulus response compatibility paradigm (i.e., faster responding to compatible than to incompatible stimuli). While there was some evidence (e.g., RT distribution analyses) for differing time courses within trials, these differences are relatively modest and can be accounted for primarily by the model's timing differences in activation of the spatial and imitative compatibilities (although the validity of these differences are questionable as discussed above). A stronger test of the sufficiency of the model is to simulate the results from an experiment yielding qualitatively different results for spatial and imitative compatibilities (Boyer et al., 2012; Sauser & Billard, 2006).

Such an experiment was proposed by Sauser and Billard (2006) to test two different neural architectures for modeling spatial and imitative compatibilities. These two models (single route and direct matching) were briefly described by CCH: the single route model, which is roughly similar to their own model, is capable of simulating both spatial and imitative compatibilities; but so is the direct matching model which differs significantly from the CCH model by incorporating a direct processing path between the observation and execution of actions. It is for this reason that Sauser and Billard (2006) conclude that stimulus response compatibility effects cannot discriminate between the two models. As a more valid test of the two different architectures, they proposed simulating the results of a variant of the stimulus response compatibility paradigm in which participants are instructed to map the perceived stimulus to the opposite response. For example, a tapping index finger would be mapped to the respondent's middle finger when instructed to respond to the anatomical identity of the finger, and a leftward tapping finger would be mapped to the middle finger of the respondent's right hand when instructed to respond to the spatial position of the two fingers (see Fig. 1, right panels). Our implementation of this opposite stimulus response compatibility paradigm is discussed in more detail by CCH when describing our first model for imitative and spatial compatibilities (Boyer, Scheutz, & Bertenthal, 2009). Unlike the standard stimulus response compatibility paradigm, the results from this paradigm reveal a reverse spatial compatibility effect (e.g., Hedge & Marsh, 1975).

Sauser and Billard (2006) were able to simulate this reverse spatial compatibility effect with their single route model, but not with their direct matching model. In contrast, both models predicted little or no reversal effect for imitative compatibility tested with an opposite stimulus response compatibility paradigm. Although these predictions initially lacked empirical confirmation, Boyer et al. (2009, 2012) confirmed that there was no reversal of the imitative compatibility effect with this paradigm. These results thus suggest that both the single route and the direct matching model are legitimate candidates for explaining imitative compatibility, but only the single route model explains spatial compatibility. Based on these results, it can be inferred that the direct matching model represents a specialized process that applies only to imitative compatibility and not to other stimulus response compatibilities.

Although these results represent a potential challenge to the claim by CCH that spatial and imitative compatibilities are mediated by the same processes, the evidence is somewhat ambiguous because there is nothing to disconfirm the validity of the single route model which explains all the results. To offer more convincing evidence that spatial and imitative compatibilities are not mediated by identical processes, we recently revised our original computational model (Boyer et al., 2009) and developed a three-layered (input-hidden-output) connectionist network to assess how well it would fit the data from Boyer et al. (2012). In particular, we tested whether the same architecture would be capable of predicting both a reversal for the spatial compatibility effect as well as no reversal for the imitative compatibility effect.

A description of the model and its success in fitting the Boyer et al. (2012) data is presented in a recent paper by Scheutz and Bertenthal (2012). Like the CCH model, our model is based on an interactive activation framework (Rumelhart, McClelland, & Group, 1986) designed to explain stimulus response compatibility effects via a dual route model consisting of an intentional and automatic route (e.g., Zhang, Zhang, & Kornblum, 1999). In spite of sharing a number of general processing assumptions (feedforward architecture, no inhibitory connections), our model differed from the one presented by CCH because it was deterministic rather than stochastic, included fewer free parameters, and included a middle decision layer for translating the input stimulus to the response. This translation process was essential to our model because the stimulus input was not always mapped to the same response; in some conditions the input was mapped to the opposite response.

Briefly, the architecture of the model includes the following nodes and connections: There are four input nodes relating to finger identity (index, middle) and spatial location (left, right), and two output nodes corresponding to the index and middle fingers of the right hand (left/index, right/middle). Input nodes are connected to middle nodes via both task-based as well as automatic connections, which, in turn, are connected to output nodes (see Fig. 2). In addition, there are direct connections between the two finger identity nodes and the corresponding output nodes (i.e., the index finger node is connected directly to the index finger response), but not between the spatial location nodes and the corresponding output nodes. Similar to the direct matching model proposed by Sauser and Billard (2006), these direct connections were inspired by research suggesting that the perception of actions automatically activates corresponding motor programs in the observer (e.g., Brass & Heyes, 2005; Jeannerod, 1994; Prinz, 1997; Rizzolatti, Fogassi, & Gallese, 2001). Thus, unlike the CCH model, our model is not symmetrical with regard to the connections mediating spatial and imitative compatibilities, and it is this asymmetry that is critical to the success of our model (see below).

The activation by these direct connections facilitated both the spatially and imitatively compatible responses in the standard stimulus response compatibility paradigm, but they facilitated only the imitatively compatible response in the stimulus response compatibility paradigm in which the opposite stimulus was mapped to the response. Unlike the original model proposed by Boyer et al. (2009), this model fit all the results, including the reverse spatial compatibility effect and the lack of a reversal for imitative compatibility. Thus, this revised model cannot be dismissed by CCH for reasons of not explaining all the results, which was their response to the previous model from 2009.

Figure 2.

The proposed neural network model for the standard stimulus response compatibility task: four input nodes consisting of Index (I) and Middle (M) fingers, and Left (L) and Right (R) spatial locations; two middle nodes with direct connections from I and L (SR-IL) or from M and R (SR-MR); and two output nodes with direct connections from I and SR-IL (IL) or from M and SR-MR (MR). The base model consists of only the dashed connections; the bold connections depict the S-R compatible tasked-based connections for the index finger.

The principal goal of this modeling research was to test the assertion by CCH that spatial and imitative compatibilities are mediated by the same process. If true, then the effects of directly mapping the connections between the finger identity and response nodes should be no different than directly mapping the connections between spatial location and response nodes. In essence, this model with direct spatial connections is simply the mirror image of the one with direct imitative connections, and indeed, fits the empirical results for the standard stimulus response compatibility effects as well as the previously described CCH model. In contrast, this model did not exhibit a good fit for the reverse compatibility effects, and thus confirmed that directly mapping finger identity versus spatial location connections are not equivalent when fitting the Boyer et al. (2012) results.

The reason for this last result is somewhat ambiguous because it could be attributable to replacing the direct imitative connections with direct spatial connections, or alternatively, it could be attributable to eliminating the direct imitative connections. Interestingly, the inclusion of both direct spatial position and finger identity connections revealed an even worse fit for the standard as well as opposite stimulus response compatibility effects than reported for the model including only direct spatial connections. In conclusion, the comparison of these different models reveals that spatial and imitative compatibilities are not mediated by exactly the same processes given that the processing of finger identity involves additional direct connections to the output nodes whereas the processing of spatial location does not.

Earlier in this commentary, we cautioned against generalizing a conclusion from modeling only one experimental paradigm. To explore the generalizability of our model, we investigated whether it could simulate some of the results from the first experiment conducted by Catmur and Heyes (2011). The design of this experiment differed from our design in that the imperative stimulus involved the discrimination of two colors (orange, purple) and was completely independent of the finger identity or spatial location of the stimulus. Thus, we added two additional color nodes to the inputs which were each connected to one of the two hidden units depending on the task instructions (i.e., orange mapped to index finger abduction, purple mapped to little finger abduction, or vice versa). The task-based connections used in our previous model were held constant, but the automatic connections were slightly increased, except for the direct input–output connections which were slightly lowered. This model resulted in a very good fit of the Catmur and Heyes (2011) results, and thus suggested that the CCH model was not unique for explaining their results. It should be noted, however, that we did not simulate all the results from Catmur and Heyes (2011, Experiment 1), because they also investigated the time course of the compatibility effect during the trial. To model these within trial timing effects, it would be necessary to change the current deterministic nature of the model to include stochasticity.

5. Concluding remarks

In the conclusion of their paper, CCH state that “the model substantiates the hypothesis that spatial and imitative compatibility effects depend on processes of the same kind” (p. 24, manuscript). Certainly, the model is successful in simulating the results from Catmur and Heyes (2011), but does fitting these data with one model give them license to claim support for generalizing their hypothesis? We are skeptical for the reasons summarized in this commentary, and we argue that generalizing beyond the results of this one study is premature because automatic imitation is more complex than described by CCH, and also because their model includes some questionable assumptions which were not tested.

In spite of our reservations about the current conclusion, we are strong advocates of the approach and encourage follow-up studies that would offer more convincing support for generalizing beyond the data from a single study. More specifically, we would like to see an investigation that goes beyond simulating results from one study and instead simulates results from multiple studies or tests alternative models based on competing theories and assumptions. The application of models to test theories is an extremely valuable and productive method in cognitive science, but the paper by CCH begs the question as to what constitutes the necessary prerequisites in model testing for concluding that a theory is either valid or invalid.

Ancillary