We applaud the efforts of Niedenthal et al. to integrate multiple processes underlying recognition of smiles in facial expression into a model that can generalize to the recognition of other emotions. However, we detect a need for greater clarity regarding the role of motor mimicry in a complex judgment task. In our paradigm for recognizing emotion in a naturalistic facial expression, a static contextual image eliciting one of eight possible emotions is presented simultaneously with a five-second facial expression that is either static or dynamic. Participants judge when the facial expression is the genuine response to the paired contextual image (“match”) and when it is not (“mismatch”) (Hamon-Hill & Barresi Reference Hamon-Hill and Barresi2008a; Reference Hamon-Hill and Barresi2008b; Reference Hamon-Hill and Barresi2009; Ouellette et al. Reference Ouellette, Hamon-Hill and Barresi2010). To test for reliance on motor mimicry during this task, participants hold a pen horizontally in their mouth throughout alternating blocks of trials. In agreement with the SIMS model, we recognize embodied simulation as a key component in the underlying processes activated during this task (Barresi & Moore Reference Barresi and Moore1996; Reference Barresi, Moore, Zlatev, Racine, Sinha and Itkonen2008). We propose the observers, while perceiving the paired stimuli, generate a first-person embodied response to the contextual image, which includes autonomic, somatic, and semantic aspects. The observers also generate a third-person representation of the target facial expression and search for a match between their own embodied response to the contextual image and their understanding of the target facial expression. For instance, observation of an elderly man holding the hand of his dying wife may elicit a response of “feeling sad,” prompting the observers to search for signs of sadness in the target facial expression.
Earlier simulation models for emotion recognition propose facial mimicry contributes to recognition of facial expressions (Goldman & Sripada Reference Goldman and Sripada2005); however, there is limited support for this claim (Blairy et al. Reference Blairy, Herrera and Hess1999; Hess & Blairy Reference Hess and Blairy2001), and imaging studies measuring motor activity seldom test for accuracy in judgments (Carr et al. Reference Carr, Iacoboni, Dubeau, Mazziotta and Lenzi2003). Using our paradigm, we tested for an effect of disruption to motor mimicry on judgments about a dynamic facial response as genuine to a given context. We tested the effect of two experimental manipulations, one to inhibit mimicry (Niedenthal et al. Reference Niedenthal, Brauer and Innes-Ker2001) and the other to interfere with mimicry (Oberman et al. Reference Oberman, Winkielman and Ramachandran2007). These two manipulations, which require holding a pen in one's mouth, were cited indiscriminately in the target article, suggesting both inhibit motor mimicry. However, Oberman et al. (Reference Oberman, Winkielman and Ramachandran2007) established a difference between the two: Whereas holding a pen between the lips and teeth (Niedenthal et al. Reference Niedenthal, Brauer and Innes-Ker2001) holds the muscles in a neutral state, thereby “inhibiting” a motor response, “biting” down on a pen held horizontally between one's teeth, without the lips touching the pen, “interferes” with spontaneous motor mimicry due to “noise” generated by consistent activation of multiple facial muscles (Oberman et al. Reference Oberman, Winkielman and Ramachandran2007). During this manipulation, activation in the motor systems is neither absent nor reduced as suggested in the target article.
Like Oberman et al. (Reference Oberman, Winkielman and Ramachandran2007), we saw no effect of disruption to mimicry on accuracy resulting from the inhibition manipulation. However, observers in the inhibition condition and the no-pen control condition both showed a significant response bias to judge the target face as a match to the paired context image. Surprisingly, the response bias was absent in the interference condition, where participants correctly discriminated when faces were not a genuine response to the paired contextual image. When we repeated the experiment applying only the interference manipulation to test for a differential effect on processing dynamic or static facial expressions, once again accuracy was not altered, but processing was affected. When mimicry was disrupted, observers were slower to process static, positive faces, yet were faster to process dynamic, positive faces.
The SIMS model offers an explanation for a response bias to facial expressions based on conceptual knowledge. According to the model, an embodied response to a context stimulus should prime the observer's judgment about the emotion perceived in a target facial expression. Additionally, disruption to motor mimicry should increase the priming effect due to greater reliance on this embodiment of conceptual knowledge. Our paradigm emphasizes the incorporation of embodied responses to context into judgments made about facial expression. However, the response bias was strongest when participants were free to mimic and while mimicry was “inhibited”, yet it was significantly diminished during “interference” with mimicry. Niedenthal et al.'s account of processing facial expression when motor mimicry is disrupted does not adequately explain what we have observed. We suggest the interference generated by the bite manipulation causes a mismatch state in the self with respect to the embodied response to the context image, and, consequently, the observer searches for a “mismatch” in the other. This leads to a reduction in the bias to see a match and increases the likelihood of improved performance on mismatch trials.
Another limitation to the model is the lack of adequate discussion regarding the quality of stimulus, in particular the difference due to static and dynamic emotional expressions. Neuroimaging studies indicate greater motor activity during observation of static compared to dynamic faces (Enticott et al. Reference Enticott, Johnston, Herring, Hoy and Fitzgerald2008; Kilts et al. Reference Kilts, Egan, Gideon, Ely and Hoffman2003). Moreover, we would expect less reliance on embodied simulation involving motor mimicry during perception of dynamic faces because of the rich quality of information available in a dynamic facial expression (Ambadar et al. Reference Ambadar, Schooler and Cohn2005). Dynamic facial displays can reveal blended emotions that change over time, shifting in degrees of expressivity. Perception of static faces, which typically present the apex of the expression, may require the observer to enrich the information by simulating the full expression. One consequence of this difference between static and dynamic expressions is that we would expect greater interference with processing static faces when motor mimicry is disrupted. That is indeed what we observed. Disruption to mimicry resulted in slower processing of static but not dynamic faces.
In sum, motor mimicry, while a contributor to a fully embodied form of emotion recognition, may not be an essential component when interpreting rich facial expressions in naturalistic contexts, but rather may primarily serve social functions such as indicating sympathy and understanding.
We applaud the efforts of Niedenthal et al. to integrate multiple processes underlying recognition of smiles in facial expression into a model that can generalize to the recognition of other emotions. However, we detect a need for greater clarity regarding the role of motor mimicry in a complex judgment task. In our paradigm for recognizing emotion in a naturalistic facial expression, a static contextual image eliciting one of eight possible emotions is presented simultaneously with a five-second facial expression that is either static or dynamic. Participants judge when the facial expression is the genuine response to the paired contextual image (“match”) and when it is not (“mismatch”) (Hamon-Hill & Barresi Reference Hamon-Hill and Barresi2008a; Reference Hamon-Hill and Barresi2008b; Reference Hamon-Hill and Barresi2009; Ouellette et al. Reference Ouellette, Hamon-Hill and Barresi2010). To test for reliance on motor mimicry during this task, participants hold a pen horizontally in their mouth throughout alternating blocks of trials. In agreement with the SIMS model, we recognize embodied simulation as a key component in the underlying processes activated during this task (Barresi & Moore Reference Barresi and Moore1996; Reference Barresi, Moore, Zlatev, Racine, Sinha and Itkonen2008). We propose the observers, while perceiving the paired stimuli, generate a first-person embodied response to the contextual image, which includes autonomic, somatic, and semantic aspects. The observers also generate a third-person representation of the target facial expression and search for a match between their own embodied response to the contextual image and their understanding of the target facial expression. For instance, observation of an elderly man holding the hand of his dying wife may elicit a response of “feeling sad,” prompting the observers to search for signs of sadness in the target facial expression.
Earlier simulation models for emotion recognition propose facial mimicry contributes to recognition of facial expressions (Goldman & Sripada Reference Goldman and Sripada2005); however, there is limited support for this claim (Blairy et al. Reference Blairy, Herrera and Hess1999; Hess & Blairy Reference Hess and Blairy2001), and imaging studies measuring motor activity seldom test for accuracy in judgments (Carr et al. Reference Carr, Iacoboni, Dubeau, Mazziotta and Lenzi2003). Using our paradigm, we tested for an effect of disruption to motor mimicry on judgments about a dynamic facial response as genuine to a given context. We tested the effect of two experimental manipulations, one to inhibit mimicry (Niedenthal et al. Reference Niedenthal, Brauer and Innes-Ker2001) and the other to interfere with mimicry (Oberman et al. Reference Oberman, Winkielman and Ramachandran2007). These two manipulations, which require holding a pen in one's mouth, were cited indiscriminately in the target article, suggesting both inhibit motor mimicry. However, Oberman et al. (Reference Oberman, Winkielman and Ramachandran2007) established a difference between the two: Whereas holding a pen between the lips and teeth (Niedenthal et al. Reference Niedenthal, Brauer and Innes-Ker2001) holds the muscles in a neutral state, thereby “inhibiting” a motor response, “biting” down on a pen held horizontally between one's teeth, without the lips touching the pen, “interferes” with spontaneous motor mimicry due to “noise” generated by consistent activation of multiple facial muscles (Oberman et al. Reference Oberman, Winkielman and Ramachandran2007). During this manipulation, activation in the motor systems is neither absent nor reduced as suggested in the target article.
Like Oberman et al. (Reference Oberman, Winkielman and Ramachandran2007), we saw no effect of disruption to mimicry on accuracy resulting from the inhibition manipulation. However, observers in the inhibition condition and the no-pen control condition both showed a significant response bias to judge the target face as a match to the paired context image. Surprisingly, the response bias was absent in the interference condition, where participants correctly discriminated when faces were not a genuine response to the paired contextual image. When we repeated the experiment applying only the interference manipulation to test for a differential effect on processing dynamic or static facial expressions, once again accuracy was not altered, but processing was affected. When mimicry was disrupted, observers were slower to process static, positive faces, yet were faster to process dynamic, positive faces.
The SIMS model offers an explanation for a response bias to facial expressions based on conceptual knowledge. According to the model, an embodied response to a context stimulus should prime the observer's judgment about the emotion perceived in a target facial expression. Additionally, disruption to motor mimicry should increase the priming effect due to greater reliance on this embodiment of conceptual knowledge. Our paradigm emphasizes the incorporation of embodied responses to context into judgments made about facial expression. However, the response bias was strongest when participants were free to mimic and while mimicry was “inhibited”, yet it was significantly diminished during “interference” with mimicry. Niedenthal et al.'s account of processing facial expression when motor mimicry is disrupted does not adequately explain what we have observed. We suggest the interference generated by the bite manipulation causes a mismatch state in the self with respect to the embodied response to the context image, and, consequently, the observer searches for a “mismatch” in the other. This leads to a reduction in the bias to see a match and increases the likelihood of improved performance on mismatch trials.
Another limitation to the model is the lack of adequate discussion regarding the quality of stimulus, in particular the difference due to static and dynamic emotional expressions. Neuroimaging studies indicate greater motor activity during observation of static compared to dynamic faces (Enticott et al. Reference Enticott, Johnston, Herring, Hoy and Fitzgerald2008; Kilts et al. Reference Kilts, Egan, Gideon, Ely and Hoffman2003). Moreover, we would expect less reliance on embodied simulation involving motor mimicry during perception of dynamic faces because of the rich quality of information available in a dynamic facial expression (Ambadar et al. Reference Ambadar, Schooler and Cohn2005). Dynamic facial displays can reveal blended emotions that change over time, shifting in degrees of expressivity. Perception of static faces, which typically present the apex of the expression, may require the observer to enrich the information by simulating the full expression. One consequence of this difference between static and dynamic expressions is that we would expect greater interference with processing static faces when motor mimicry is disrupted. That is indeed what we observed. Disruption to mimicry resulted in slower processing of static but not dynamic faces.
In sum, motor mimicry, while a contributor to a fully embodied form of emotion recognition, may not be an essential component when interpreting rich facial expressions in naturalistic contexts, but rather may primarily serve social functions such as indicating sympathy and understanding.