SEARCH

SEARCH BY CITATION

Keywords:

  • action observation;
  • language;
  • premotor cortex;
  • transcranial magnetic stimulation

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

Previous studies using functional magnetic resonance imaging and transcranial magnetic stimulation (TMS) explored the relationships between linguistic processing and motor resonance, i.e. the activation of the motor system while perceiving others performing an action. These studies have mainly investigated a specific linguistic domain, i.e. semantics, whereas phonology has been largely neglected. Here we used single-pulse TMS to compare the effects of semantic and phonological processing with motor resonance effects. We applied TMS to the primary motor hand area while subjects observed object-oriented actions and performed semantic and phonological tasks related to the observed action. Motor evoked potentials were recorded in two hand muscles, one of them more involved in the execution of the observed actions than the other one, at three different timepoints (0, 200 and 400 ms after stimulus onset). The results demonstrated increased corticospinal excitability that was muscle-specific (i.e. restricted to the hand muscle involved in the observed action), hemisphere-specific (left), and time-specific (400 ms after stimulus onset). The results suggest an additive effect of independent semantic and phonological processing on motor resonance. The novel phonological effect reported here expands the links between language and the motor system and is consistent with a theory of shared control for hand and mouth. Furthermore, the timing of the semantic effect suggests that motor activation during semantic processing is not an ‘epiphenomenon’ but rather is essential to the construction of meaning.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

Two major frameworks of cognitive operations have typically been pitted against each other. A ‘classical cognitivism’ view assumes that cognitive processes are amodal, abstract, and rule-based manipulations of symbols (Newell, 1990). In contrast, the ‘embodied cognition’ framework assumes that cognition is grounded in the perceptual and motor experiences of the body (Anderson, 2003). Although these two frameworks can be traced all the way back to Plato and Aristotle, they have been explored extensively by recent empirical research in the cognitive neurosciences.

A frequently used approach to the study of embodied cognition is the investigation of the relationships between language (a cognitive function par excellence) and the motor system (Glenberg & Kaschak, 2002; Gallese & Lakoff, 2005; Pulvermüller, 2005). This line of research is inspired by the theory of embodied semantics (Gallese & Lakoff, 2005; among many others) and by motor theories of speech perception (Liberman & Mattingly, 1985). The former assumes that motor representations are essential components of certain concepts, especially concepts about actions. The latter assumes that the motor system is actively involved in speech perception. The underlying functional mechanism of both theories is a neural mechanism of mirroring. Concepts about actions can be formed and understood through a reactivation of motor plans used to execute those actions, whereas speech sounds can be perceived through the subthreshold activation of the motor, articulatory acts that would be necessary to emit those speech sounds.

The work on language and action has been recently inspired by the discovery of mirror neurons (Gallese et al., 1996) on two main grounds. First, these cells were originally recorded in area F5 of the monkey brain, an area that may be the monkey homolog of the posterior part of the human Broca’s area, a major language area. Second, by activating during both the production and perception of action, these cells seem to provide a common code between the sender and the receiver of a message, which is considered a necessary functional mechanism for communication to occur (Rizzolatti & Arbib, 1998).

The roots of the empirical work on mirror neurons, i.e. the neurophysiological investigation of the ventral premotor cortex in macaques, had suggested possible common mechanisms of control for hand and mouth (see, e.g. Rizzolatti et al., 1988). Hand and mouth neurons co-exist in the ventral premotor cortex, and some neurons code for ‘hand-to-mouth’ actions. This hypothesis has been subsequently systematically investigated with behavioral studies that provided compelling evidence in its support (see, e.g. Gentilucci et al., 2008). Thus, there are now at least three main theoretical frameworks that can guide empirical research on the relationships between language and motor structures.

Here we used single-pulse transcranial magnetic stimulation (TMS) to investigate the potential influence of semantic and phonological processing on motor corticospinal excitability. Subjects observed actions directed at graspable objects or just the objects (with no actions directed at them) while they performed semantic, phonological and perceptual decisions on the stimuli. Modulation of motor corticospinal excitability, which is enhanced during action observation (see, e.g. Fadiga et al., 1995), was taken as an index of the influence of semantic and phonological processing on motor activity. Furthermore, with single-pulse TMS, we explored the timing of motor facilitation at different timepoints. This allowed us to investigate the functional role and timecourse of motor facilitation in cognitive tasks requiring linguistic (semantic and phonological) processing. On the basis of previous studies on linguistic processing (Barett & Rugg, 1990; Kiefer, 2001; Federmeier & Kutas, 2002; Vihla et al., 2006), we chose 200 and 400 ms after stimulus onset as timepoints of interest. As control, the start of the stimulus presentation (0 ms) was also investigated.

We recorded motor evoked potentials (MEPs) from two hand muscles [first dorsal interosseus (FDI) and abductor pollicis brevis (APB)]. In a pilot study, we found differential involvement of these two hand muscles when an independent set of subjects executed the actions that were used for our main TMS experiment.

Materials and methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

We studied 13 healthy right-handed native English speakers [age 26.5 ± 3.08 (SEM) years, six men].

Ethics statement

The study was approved by the Institutional Review Board of the University of California at Los Angeles and all subjects gave written informed consent for participation in this study. All investigations were conducted according to the principles expressed in the Declaration of Helsinki.

Task and stimuli

Stimuli consisted of 30 manipulable objects, presented either alone in a picture or in short videos showing a hand acting on the object. For instance, there was a picture of a light bulb and a video of someone’s hand screwing in the light bulb. The list of the objects used for the experimental set of stimuli is provided in Appendix 1. In the videos, the objects always were manipulated with the right hand and a third-person perspective was adopted. The subjects’ task was to decide whether:

  • (i)
     the object presented was a tool [Object–Semantic (Obj–Sem) and Action–Semantic (Act–Sem) categorization task];
  • (ii)
     the word best describing the object contained the diphthong “/aΙ/” (as in “dice”) [Object–Phonological (Obj–Phon) and Action–Phonological (Act–Phon) task];
  • (iii)
     all five fingers of the grasping hand touched the object (this task was applied to action videos only) [Action–Observation (Act–Obs) task]; and
  • (iv)
     any visible part of the object was black (this task was applied to object pictures only) [Object–Observation (Obj–Obs) task].

Subjects were asked to respond with button presses as quickly as possible (see below). An overview of the experimental design is given in Fig. 1.

image

Figure 1.  Schematic overview of the experimental design of the TMS experiment. A total of six experimental conditions were tested, three conditions requiring the observation of videos of everyday hand actions directed towards objects, and the other three conditions involving perception of photos of the same objects. Every stimulus class was related to an observational decision task, a semantic categorization decision task, and a phonological decision task. MEPs were recorded during three different timepoints (0, 200 or 400 ms after stimulus onset) from two different hand muscles in both hands. This design allowed a characterization of changes in motor cortex excitability during action observation and linguistic tasks regarding the timecourse and functional-anatomic specificity in terms of hemisphere and cortical movement representation.

Download figure to PowerPoint

The tasks required subjects to perform semantic categorization and phonological decisions on pictures (Obj–Sem and Obj–Phon) and videos (Act–Sem and Act–Phon), and to perform two additional decisions based on object (Obj–Obs) or action features (Act–Obs). This experimental design allowed the study of the effects of action observation and linguistic (semantic or phonological) processing on hand motor cortex excitability, and allowed testing of whether action observation and linguistic processing have additive effects on such excitability. Indeed, if this is the case, we anticipate that measures of motor cortex excitability (see below for details) should be higher for tasks involving concurrent action observation and semantic/phonological processing than for tasks that solely involve either action observation or semantic/phonological processing alone.

An independent group of 10 healthy native English-speaking subjects who did not participate in the TMS study was tested to select the set of experimental stimuli. Subjects were shown 60 pictures of manipulable objects. Subjects were asked two questions: whether the objects presented were common tools and which word they would choose to describe each object. Thirty objects that elicited consistent answers in all 10 subjects were selected for the TMS experiment. A static picture of the object on a plain background and a video showing someone’s hand manipulating the object were created for each of these 30 objects. Both picture and video stimuli lasted 3 s. The same set of stimuli was used for all tasks. Furthermore, the 30 stimuli chosen for the experiment were selected such that correct decisions made in each task condition would yield a ‘yes’ response for 9 stimuli and a ‘no’ response for 21 stimuli. The responses required for each task were independent, i.e. a stimulus that required a ‘yes’ response for one task did not necessarily require a ‘yes’ response for the other tasks. For each of the 36 conditions (6 tasks × 3 timepoints × 2 hemispheres), 12 stimuli out of the whole set of 30 stimuli were chosen in such a way that the number of required ‘yes’ and ‘no’ responses was constant across conditions.

We designed the TMS study to investigate how action observation and linguistic processing affect the hand motor cortex, especially with regard to the temporal and functional anatomic specificity of the hand motor cortex during these tasks. Therefore, TMS was used to elicit MEPs in two different hand muscles, the FDI and APB muscle, which are differentially involved in the execution of the object-oriented actions presented to subjects during the experiment. To test and measure the differential involvement of these two hand muscles in action execution, we used surface electromyography in a pilot study in a different group of eight subjects (four men). After watching each one of the video stimuli used for the main experiment, subjects were asked to execute the observed action on the objects. Electromyography (EMG) signals from the APB and FDI were recorded with a surface EMG system (Bagnoli-4; Delsys, Inc.) with a bandwidth of 20–450 Hz. Data were sampled at 2000 Hz and analysed offline using a custom program written in MATLAB (dataWizard). For each EMG signal, an EMG envelope was extracted by rectifying and filtering each signal with a low-pass, zero-lag, Butterworth filter with a 5 Hz cutoff frequency (Fig. 2). The average absolute value of the EMG envelope slope was taken as a measure of the phasic activity for that EMG channel. As slopes are dependent on the amplitude of EMG bursts, all EMG envelopes were normalized to the maximum voluntary contraction for that muscle in each subject. The normalized values obtained through this data analysis procedure (indexing the degree of phasic activity for both FDI and APB muscles) were statistically compared using a nonparametric Wilcoxon test due to the small number of subjects included.

image

Figure 2.  Data analysis of the surface EMG data during the behavioral study. During execution of the object-directed hand actions that were later tested in the TMS study, surface EMG data from two hand muscles (APB and FDI) were recorded (the left part of the figure shows data for the action of screwing in a bulb in a representative subject). EMG data were then rectified and an envelope curve was computed (right). The average slope of this envelope curve was taken as an indicator of EMG variation during the action, i.e. phasic EMG activity.

Download figure to PowerPoint

Phasic muscle activity, as measured by variations of the EMG pattern, was significantly greater in the APB compared with the FDI muscle (APB, 0.97 ± 0.33; FDI, 0.55 ± 0.13 normalized values, Wilcoxon rank sum test, = 0.036) during execution of the same actions as displayed to subjects in the TMS experiment. This overall greater phasic involvement of the APB during action execution for the set of actions used in the TMS experiment in this pilot study (see Fig. 1) may predict a higher motor resonance for the APB in the TMS experiment.

Transcranial magnetic stimulation

Motor evoked potentials

Transcranial magnetic stimulation was applied with a Super Rapid stimulator (Magstim Co., UK) using a figure-of-eight coil. The coil was placed on the scalp and the site of stimulation was adjusted in order to elicit MEPs with maximum amplitude in two contralateral hand muscles. During the experiment, we simultaneously recorded MEPs from the contralateral FDI and APB muscles using surface Ag/AgCl electrodes placed over each muscle belly and a metacarpophalangeal joint (reference electrode). The TMS stimulus site was first adjusted to obtain maximal and consistent MEP output from both muscles during combined recording. The lowest stimulus intensity was then determined, which evoked MEPs with > 50 μV peak-to-peak amplitude in at least 5 out of 10 trials in both relaxed FDI and APB muscles during simultaneous recording. The stimulus intensity used for the TMS experiment was 120% of the intensity determined in this way for each hemisphere. Subjects were instructed to keep their hands still but relaxed during the experiment. The background EMG activity of the hand muscles was monitored 100 ms before single MEP trials to control for involuntary hand movements; no trial had to be discarded due to insufficient hand relaxation.

Procedure

Subjects sat in a reclining chair and were instructed to watch the 21-inch computer screen, which was located 60 cm away. We delivered TMS and recorded MEPs at three different timepoints: 0, 200 and 400 ms after stimulus onset. The experiment was composed of a total of six blocks of trials (three timepoints for each motor cortex), the order of which was randomized across subjects. The timepoints were chosen on the basis of published data on temporal dynamics of linguistic processing (Barett & Rugg, 1990; Kiefer, 2001; Federmeier & Kutas, 2002; Vihla et al., 2006). These studies show that the stimulus analysis and early processes related to phonological and semantic access are taking place in left temporoparietal and frontal regions between about 150 and 250 ms; furthermore, there is a later component of linguistic processing between about 300 and 450 ms, which is thought to represent converging articulatory and phonological processes with a focus of activity on left frontal regions. A few studies on the temporal neural dynamics of action observation support a similar timecourse of neural activation (Nishitani & Hari, 2000; Hauk et al., 2008). The timepoint of 0 ms at stimulus onset was chosen as control.

Results of previous TMS studies suggest that there might be cyclic facilitation changes within the APB and FDI during observation of repetitive hand movements occurring during several actions investigated in our study (i.e. screwing in a bulb) (Gangitano et al., 2001; Urgesi et al., 2006). In our design, however, this issue was not a factor. Indeed, the timepoints of TMS stimulation were early enough that subjects were only watching someone’s hand appearing on the screen, then approaching the object and grasping it. No TMS pulse was delivered while the rest of the action unfolded (e.g. screwing in the light bulb). Indeed, the rationale for the early timepoints for TMS stimulation was largely based on the Event-related potential literature on linguistic processing.

Subjects were tested first with TMS applied to the motor cortex over one hemisphere followed by the other hemisphere. The first hemisphere tested was randomized across subjects. For each hemisphere, subjects performed 1 run consisting of 36 condition blocks. Each condition block consisted of six test trials, all of one task type (Act–Obs, Act–Sem, Act–Phon, Obj–Obs, Obj–Sem, Obj–Phon). Each condition block was preceded by a visual cue consisting of a text display for 2 s, instructing the subject on the type of task to perform for the next six test trials. For each test trial, TMS was delivered at 0, 200, or 400 ms.

The order of condition blocks was randomized across subjects. Within each condition block, the order of stimuli was the same for all subjects. Each run (hemisphere) therefore resulted in 216 test trials, with a balanced distribution of 12 test trials for each of 3 TMS timepoints and 6 task conditions. In addition, each test trial resulted in both APB and FDI MEPs. All video and photo stimuli were presented for 3 s; the time interval between stimuli was 2 s, thus TMS was applied approximately every 5 s. We assume that the 5 s interstimulus interval did not influence MEP amplitudes during the experiment, as there is no evidence for such a net effect and 5 s is an interval considered to be a single-pulse TMS (Wassermann et al. 1998). Furthermore, the sequence of experimental conditions was pseudorandomized to prevent order effects.

All videos displayed actions of the same duration, which were carried out with the right hand. Subjects were instructed to respond with the middle finger of the hand contralateral to the one from which MEPs were recorded. Thus, motor responses did not involve APB and FDI muscles. Furthermore, a potential role of the motor response in influencing MEPs should have affected all tasks similarly, and is thus unlikely to have influenced the statistically reliable effects reported here.

Data analysis

The MEP data obtained for all experimental conditions were processed using a custom software package (dataWizard, A.D.W.) (described in detail in Koski et al., 2005) written in matlab (Mathworks Co.). Peak-to-peak amplitudes were extracted for further analysis from each single sweep. Individual single MEP amplitude values were averaged per experimental condition. Individual MEP values that differed by more than 2 SDs from the mean were discarded. Roughly 2% of the MEP data had to be discarded through this correction procedure. The average MEP amplitudes for each subject and experimental condition obtained after this preprocessing procedure were analysed with repeated-measure anova. There was a total of 72 experimental conditions for a 2 × 2 × 3 × 6 factorial design with the within-subject factors ‘hemisphere’ (left vs. right), ‘muscle’ (FDI vs. APB), ‘time’ (0, 200, 400 ms after stimulus onset) and ‘task’ (Obj–Obs, Obj–Phon, Obj–Sem, Act–Obs, Act–Phon, Act–Sem). Post hoc t-tests were planned for significant main effects and interactions.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

The overall average correctness rate for all tasks of the main experiment was 85.5 ± 7.6% (an overview of the number of correct responses depending on the task is given in Fig. S1). The overall average reaction time was 1002.8 ± 10.8 ms. Analysis of the reaction time data and the number of correct responses using multivariate within-subject anovas with the main factors ‘task’, ‘hemisphere’, and ‘timepoint’ did not reveal significant main effects or interactions. Thus, TMS stimulation of the hand motor cortex did not influence performance differentially across hemispheres or timepoints.

Average MEP amplitudes for left hemisphere stimulation were 0.60 ± 0.04 mV (APB) and 0.91 ± 0.04 mV (FDI), and, for right hemisphere stimulation, average MEP amplitudes were 0.60 ± 0.03 mV (APB) and 0.95 ± 0.03 mV (FDI). To determine if MEP amplitudes were determined by reaction times, a regression analysis between average MEP values and reaction time values was run for the APB and FDI separately. This analysis did not show significant effects, thus differences in reaction times across tasks did not influence MEP amplitudes.

Repeated-measures anova with hemisphere, muscle, stimulation time, and task as within-subject variables and MEP as dependent variable showed a main effect of the factor ‘muscle’ (df = 1, 9; F = 11.138; P = 0.01), as the average MEP amplitudes recorded over the APB muscle were smaller than the amplitudes recorded over the FDI muscle. However, as the average amplitudes obtained in the present study for both FDI and APB were in a range well below the possible maximum amplitude, it is highly unlikely that the higher amplitude of FDI MEP led to saturation or ceiling effects.

Furthermore, a significant interaction of all four experimental factors (hemisphere × muscle × time × task interaction, significant within-subject effect, df = 10, 90; F = 2.265; = 0.02) was found. All other main effects and interactions between the experimental factors (including the interaction of the factors ‘muscle’ and ‘task’) were not significant (main effect ‘time’: df = 2, = 0.105, P = 0.9; main effect ‘task’: df = 5, F = 0.679, P = 0.64; main effect ‘hemisphere’: df = 1, F = 0.066, P = 0.8; interaction ‘hemisphere × time’: df = 2, F = 2.742, P = 0.09; interaction ‘hemisphere × muscle’: df = 1, F = 0.066, P = 0.8; interaction ‘time × muscle’: df = 2, = 0.458, P = 0.64; interaction ‘hemisphere × task’: df = 5, F = 0.835, P = 0.53; interaction ‘time × task’: df = 10, F = 1.828, P = 0.07; interaction ‘muscle × task’: df = 5, F = 1.095, P = 0.38; interaction ‘hemisphere × time × muscle’: df = 2, = 0.516, P = 0.61; interaction ‘hemisphere × time × task’: df = 10, F = 0.555, P = 0.85; interaction ‘hemisphere × muscle × task’: df = 5, = 0.455, P = 0.81; interaction ‘time × muscle × task’: df = 10, F = 1.578, P = 0.13). To investigate the nature of the four-way interaction, subsequent anovas were performed separately for each hemisphere with the within-subject factors ‘time’, ‘muscle’ and ‘task’. They revealed a significant time by muscle by task interaction in the left hemisphere (df = 10, F = 2.042, P = 0.04) but not in the right hemisphere. Analysis of the MEP data of the right hemisphere showed only a main effect of muscle but no further main effects or double interactions. Thus, further anovas were performed only on MEPs obtained by stimulating the left hemisphere. Separate anovas in each muscle with the within-subject factors ‘time’ and ‘task’ showed a significant time by task interaction for the APB data (df = 10, F = 1.966, P = 0.04) but not for the FDI data. Thus, further anovas were performed only on APB data. One-way anovas on the left hemisphere APB data, at each timepoint separately, tested the factor ‘task’. These analyses revealed a significant effect of task for the timepoint 400 ms after stimulus onset (df = 5, F = 4.953, P = 0.001) but not for the timepoints 200 ms after stimulus onset and at stimulus onset (0 ms). Lastly, Student’s t-tests were performed to compare MEPs during the experimental tasks at 400 ms after stimulus onset. As results of this analysis were found to be significant in the stepwise anova procedure described above, no correction for multiple comparisons was carried out. Although this approach may not be the most conservative one, the clear results of the stepwise interaction analyses allow sound conclusions given the large number of conditions of the whole experiment (see also Rothman, 1990).

Pairwise statistical comparison of the average MEP amplitudes of the APB muscles 400 ms after stimulus onset for all six experimental conditions using Student’s t-test demonstrated a significant increase of excitability (P < 0.05) of the left APB for all experimental conditions compared with the Obj–Obs task, which can be conceived as a high-level baseline task in this experimental design. Critically, there was also increased APB excitability during Act–Phon (phonological processing of observed actions) compared with Act–Obs (P = 0.03, one-tailed), Act–Sem (P = 0.01, one-tailed) and Obj–Sem (P = 0.03, one-tailed) tasks. In contrast, the comparison of Act–Phon with Obj–Phon tasks (P = 0.25, one-tailed) and comparison of Obj–Phon with Obj–Sem tasks (P = 0.15, one-tailed) for the APB 400 ms left hemispheric TMS condition did not show a significant difference of MEP amplitudes. An anova analysis of the factors ‘time’ and ‘task’ for left hemispheric MEP data recorded from the FDI did not show significant main effects or interactions. Therefore, the modulation of MEP amplitudes was specific for the APB muscle. An overview of the MEP results of the different experimental tasks relative to baseline is given in Table 1.

Table 1.   Overview of the relative change of MEP amplitudes (e.g. 0.42 indicates a 42% increase, −0.12 indicates a 12% decrease) during the five experimental conditions relative to the baseline conditions for each timepoint, both muscles and hemispheres
Left hemisphereRight hemisphere
 Obj–PhonObj–SemAct–ObsAct–PhonAct–Sem Obj–PhonObj–SemAct–ObsAct–PhonAct–Sem
APBAPB
0 ms−0.08437−0.056960.0247850.2219810.2190570 ms0.1385950.1881420.1297260.215330.0415
200 ms−0.014220.025076−0.015690.096140.070106200 ms0.1357960.1190640.1030270.148272−0.07606
400 ms0.9749570.4266190.3197860.7379920.298901400 ms0.1864010.006910.3191190.3164190.157261
FDIFDI
0 ms−0.051230.0998980.2043720.1324050.1727150 ms0.020503−0.08087−0.00709−0.09975−0.03704
200 ms0.073342−0.000280.0943850.2190310.190639200 ms−0.06169−0.09823−0.12401−0.05875−0.07099
400 ms0.5856960.2027380.3443120.1850160.237351400 ms0.0759950.0328020.181048−0.122340.059232

Taken together, both linguistic processes and action observation produced increased excitability of the hand motor cortex, which was specific for the timepoint of 400 ms, the left hemisphere and the representation of the muscle, which was mainly involved in manipulation of the objects used as stimuli. Furthermore, the facilitatory effect of the phonological decision task was higher than the effect of the other two tasks. MEP data of the right FDI and APB during the timepoint of 400 ms are shown in Figs 3 and 4.

image

Figure 3. anova analysis of the MEP data of the whole data set shows a hemisphere-specific, timepoint-specific and muscle-specific increase of hand motor cortex excitability during observation of an object-directed hand action, semantic categorization or phonological decision on this object. The significant interaction of the factors ‘timepoint of MEP recording’, ‘hemisphere tested’, ‘task’ and ‘hand muscle tested’ was driven by the differences across tasks for TMS applied to the left hemisphere at 400 ms after stimulus onset and MEP recording in the right APB muscle. Average MEP amplitudes are shown (error bars indicate SE). The five experimental conditions comprising linguistic processing of action observation showed a significant increase of MEP amplitudes compared with the baseline condition Obj–Obs (indicated with one or two asterisks beside the columns). The phonological decision task on the video stimuli furthermore elicited a higher increase of hand motor cortex excitability than the other two conditions involving video stimuli and the Obj–Sem task (indicated by asterisks), showing an inherent coupling between networks for phonological processing and motor regions. **≤ 0.0056, *P < 0.05 (Student’s t-test, one-tailed).

Download figure to PowerPoint

image

Figure 4.  Average MEP amplitudes of the experimental conditions at timepoint 400 ms for recording in the FDI muscle of the right hand. In contrast to the highly significant effect of the linguistic and action observation conditions on the MEP amplitudes in the right APB muscle (Fig. 3), there were no significant effects of these conditions in the FDI muscle, which is not as critical as the APB muscle for the hand–object interactions shown on the video stimuli used for the experiment (see also Fig. 2).

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

We used TMS to test the early timecourse of semantic and phonological processing effects on motor resonance mechanisms, i.e. on facilitation of the motor system during action observation. We tested the effects on two muscles (the APB and FDI) that are differentially activated by the actions observed by our subjects. We found semantic and phonological effects only in the muscle significantly more involved in the observed actions, i.e. the APB. These effects were also time-specific, because they occurred only at 400 ms from stimulus onset, and were only observed in the left hemisphere, which is the cerebral hemisphere with higher linguistic competence.

These results expand on previous reports on the functional relation between the motor system and linguistic processes, and support existing theories on the common grounding of linguistic and action processing in the brain (see, e.g. Rizzolatti & Arbib, 1998; Glenberg & Kaschak, 2002; Gallese & Lakoff, 2005; Pulvermüller, 2005; Csibra, 2007; Pulvermüller & Fadiga, 2010). Moreover, the experimental findings reported here have at least two major theoretical implications. The first one is related to the theory of embodied semantics and its growing literature (Aziz-Zadeh et al., 2006; Reithler et al., 2007; Urgesi et al., 2007; Aziz-Zadeh & Damasio, 2008; Aziz-Zadeh & Ivry, 2009; Grafton, 2009; Buxbaum & Kalénine, 2010; Fernandino & Iacoboni, 2010; Willems et al., 2010; Arévalo et al., 2012; Pulvermüller & Fadiga, 2010). Three main types of studies have investigated the predictions of the embodied semantic theory. These studies use behavioral investigations (in both healthy subjects and neurological patients) (see, e.g. Glenberg & Kaschak, 2002; Borghi et al., 2010; Arévalo et al., 2012), brain imaging (see, e.g. Aziz-Zadeh et al., 2006; Hauk et al., 2004), or brain stimulation (see, e.g. Pitcher et al., 2008; Buccino et al., 2005). The results from this vast literature generally support the embodied semantic framework. The evidence from these studies, however, can hardly be considered conclusive. One of the major interpretive challenges regarding the activation of motor structures during semantic processing of action-related material (whether linguistic material such as sentences or visual stimuli such as those adopted in our study here) is related to its causal role in the semantic process itself. In principle, the activation of motor structures and motor representation of the actions that are semantically processed may simply be an epiphenomenon. The argument is as follows. Let’s discuss the example of the reading of a sentence describing an action. There may be an ‘amodal’, traditionally disembodied understanding of the action described linguistically that does not require activation of motor structures. The activation of the concept of the action, however, subsequently automatically (but epiphenomenally) triggers the motor representation of the action described linguistically. This hypothesis accepts the activation of motor structures while processing linguistic material associated with actions, but argues that such activation is an epiphenomenon.

A way of experimentally addressing this issue is to look at the timing of activation of motor structures. If this timing of activation is early enough and similar to the timing of activation in other language tasks, it is unlikely that the activation in motor structures while semantically processing actions that are either described linguistically or presented visually as in this study is an epiphenomenon. Our results clearly support this view. The effect that we reported here at 400 ms after stimulus onset is early enough (and corresponds well with the timing of neurophysiological effects observed in language tasks) and suggests that the involvement of the motor system in simulating the semantically processed action is essential to the semantic process itself. Indeed, an impressive range of stimuli have evoked brain responses related to semantic processing around 400 ms (Kutas & Federmeier, 2011), exactly the same temporal window as our TMS effect. This suggests that, indeed, the motor activation participates in the brain activity constructing meaning, rather than simply being an epiphenomenal, cascade-like activation triggered by an amodal representational process.

The other important theoretical implication of this study is related to the novel experimental effect reported here regarding phonological processing. Although phonological effects in TMS studies have previously been reported for speech listening tasks (Roy et al., 2008), to the best of our knowledge such effects have not been reported for tasks associated with action recognition. Indeed, our results show that, with respect to the high-level baseline task of observing the object alone, all the remaining experimental tasks demonstrated reliably higher MEP amplitudes, indexing increased motor corticospinal excitability. Furthermore, the phonological task on the observed action produced higher MEPs than the two conceptual tasks (on action and object), and the action observation task only. Taken together, these findings can be almost fully accounted for by the additive effect of semantic and phonological processing over the motor system. We propose that the three action tasks (action observation, phonological and semantic) automatically activate semantic representations of the observed actions, regardless of the task performed. In addition to this semantic effect, the phonological task on the action also activates a phonological effect on the motor cortex. Hence, the additional MEP increase, compared with the other two action tasks. The two linguistic tasks on the objects also activate, respectively, a semantic effect (the conceptual task), and a phonological effect (the phonological task), compared with the observation of the object alone.

The magnitude of the semantic and phonological effects over the motor cortex seems roughly equivalent. A pure additive model for roughly equivalent effects would explain all the results that we obtained, except one. The direct comparison between the phonological task on the action and the phonological task on the object does not yield a reliable difference. This, however, may simply be a failure in reaching a threshold for significance.

An obviously important question is why the phonological task would have an effect on the motor cortex hand area. Empirical work suggests shared motor control for the mouth and hand (Gentilucci et al., 2008). Further experimental evidence compatible with common neural substrates for phonological and manual abilities comes from a number of studies. For instance, a study using direct stimulation of the cortex demonstrated that disruption of left dorsal premotor cortex function (which is assumed to mainly represent hand movements) causes anomia (Duffau et al., 2003). Another study demonstrated that lexical decisions on hand and leg actions are speeded up by TMS over the hand and leg motor cortex (Pulvermüller et al., 2005). Furthermore, a study that screened linguistic and manual abilities in college students reported a correlation between the degree of phonemic awareness and manual skills (Carello et al., 2002).

With regard to the timing of phonological effects, they are typically seen very early for visual word recognition, but they also seem to be mediated by speech production codes (Wheat et al., 2010), thus again involving the motor system in the process. With regard to phonological effects triggered by nonlinguistic stimuli, evidence points to quite late activations (around 900 ms) in visual areas (Vihla et al., 2006). The TMS effects seen in this study would precede those activations, thus supporting the hypothesis that the novel phonological effect reported here is essential to the phonological process invoked by the task.

If we want to posit a parsimonious, unifying mechanism of simulation for the involvement of motor structures in language, then we have to assume that the phonological task may have triggered the simulation of the articulatory motor plan necessary to emit the phoneme, thus activating the mouth motor area. This activation would also spread to the hand motor area, according to the predictions of the model assuming shared control of hand and mouth.

One could argue that the hemispheric lateralization of the MEP effects described here was related to the fact that the videos showed only hand–object interactions with the right hand. As such, we acknowledge that we cannot fully exclude the possibility that the left hemispheric effects that we found could be due to an asymmetrical interhemispheric effect resulting from motor preparation in our right-handed subjects. However, we feel that this is unlikely as it would not explain the leftward lateralized increase of the MEP amplitudes in the conditions Obj–Phon and Obj–Sem, which comprised the presentation of object photos without a grasping hand. Furthermore, it may be argued that the fact that all participants of the study were right-handed could account for the results of the present study. However, an effect of handedness on the results would not explain the relative increase of MEP amplitudes in the experimental conditions involving action observation and/or linguistic decisions compared with the task Obj–Obs 400 ms after stimulus onset in the right APB. Therefore, we consider it much more likely that the lateralized increase of the MEP amplitudes is related to the left-hemispheric dominance for linguistic processing.

To summarize, we report here specific TMS experimental effects on motor activation related to semantic and phonological processing of actions. The semantic effects are in line with previously reported findings interpreted in the literature as supporting the embodied cognition model of semantic processing. The novel aspect of these findings on semantic processing is that the early timing of the effects, and their temporal coincidence with a variety of other semantic effects reported in the literature, suggests that motor activation during semantic processing is not epiphenomenal but rather causally related to the semantic process itself. The novel phonological effects reported here expand the experimentally tested links between language and the motor system, and are interpreted within a framework of shared control for hand and mouth.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

For generous support the authors wish to thank the Brain Mapping Medical Research Organization, Brain Mapping Support Foundation, Pierson-Lovelace Foundation, The Ahmanson Foundation, William M. and Linda R. Dietel Philanthropic Fund at the Northern Piedmont Community Foundation, Tamkin Foundation, Jennifer Jones-Simon Foundation, Capital Group Companies Charitable Foundation, Robson Family and Northstar Fund. The project described was supported by Grant Numbers MH68630 from the National Institute of Mental Health, and RR12169, RR13642 and RR00865 from the National Center for Research Resources, a component of the National Institutes of Health; its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institute of Mental Health, National Center for Research Resources or National Institutes of Health. A.D.W. is supported by NINDS K23-NS045764. I.G.M. was funded by the Deutsche Forschungsgemeinschaft (Grant ME 2104/3-1).

Abbreviations
Act–Obs

Action–Observation

Act–Phon

Action–Phonological

Act–Sem

Action–Semantic

APB

abductor pollicis brevis

EMG

Electromyography

FDI

first dorsal interosseus

MEP

motor evoked potential

Obj–Obs

Object–Observation

Obj–Phon

Object–Phonological

Obj–Sem

Object–Semantic

TMS

transcranial magnetic stimulation

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

Appendix

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

Appendix 1

List of the objects that were used for creating the stimulus pictures and photos for all experimental tasks.

Axe

Bell

Brush

Bulb

Calculator

Chalk

Cup

Dice

Eraser

Flashlight

Frisbee

Guitar

Hammer

Key

Ladle

Lighter

Pencil

Remote

Saltcellar

Scissors

Screwdriver

Sponge

Spoon

Spray

Stamp

Stopwatch

Syringe

Tongs

Tweezers

Typewriter

Supporting Information

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Appendix
  10. Supporting Information

Fig. S1. Number of correct responses (out of 12) for each of the experimental tasks used in the main experiment. Multivariate anova analysis of the data with the main factors ‘task’, ‘timepoint’, ‘hemisphere’ did not reveal significant main effects or interactions.

As a service to our authors and readers, this journal provides supporting information supplied by the authors. Such materials are peer-reviewed and may be re-organized for online delivery, but are not copy-edited or typeset by Wiley-Blackwell. Technical support issues arising from supporting information (other than missing files) should be addressed to the authors.

FilenameFormatSizeDescription
EJN_8134_sm_FigS1.tif996KSupporting info item

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.