We congratulate Niedenthal et al. for their outstanding contribution to both empirical and conceptual approaches in psychology and neuroscience. As reported by the authors, smiling faces are judged happier when gazing at the participant as compared to gazing sideways (e.g., Adams & Kleck Reference Adams and Kleck2005). The authors advance that such an increase in positive intensity could reflect a particularly accurate embodied simulation during smile perception. Here, we explore whether such a model could account for recognition of all emotions. Alternatively, the triggering function of eye contact may be specific to a subset of emotions that includes happiness.

Although facial expressions are efficient emotional signals, eye gaze direction is important for signaling the referent of an expression and, therefore, the attended object of the elicited emotion (George & Conty Reference George and Conty2008). Converging evidence suggests that these two signals are integrated during the perception of facial emotion. Further, behavioral studies have reported that the perception of facial expressions can be modulated by eye-gaze direction. However, the effect of gaze on emotion recognition depends on the type of expression. For example, it has been shown that angry faces are perceived to express more anger with direct than averted gaze, whereas fearful faces are perceived to express more fear with averted than direct gaze (Adams & Kleck Reference Adams and Kleck2003; Sander et al. Reference Sander, Grandjean, Kaiser, Wehrle and Scherer2007). These results can be explained within a self-relevance framework (Sander et al. Reference Sander, Grafman and Zalla2003). Indeed, it has been proposed that angry expressions with averted gaze or fearful expressions with direct gaze have less relevance for oneself than these expressions with a direct or averted gaze, respectively. The rationale is that the aversive dimension of anger is higher when one is the target of it (and hence gazed at); similarly, the threatening content of fearful expressions is higher when the face is looking away from the observer, as the object of fear for another agent can also reflect a potential menace for the observer. So far, these results suggest that emotion perception is influenced by the self-relevance of expression based on gaze direction, and this process appears to be emotion-dependent.

Niedenthal et al. propose that the amygdala may produce states that further contribute to simulations that underlie how smiles are interpreted. A recent brain imaging experiment is particularly relevant to this view and the prediction for the role of gaze contact in the processing of happiness: Sato et al. (Reference Sato, Kochiyama, Uono and Yoshikawa2010) showed that dynamic happy faces elicit more amygdala activation when gazing at–as opposed to away from–the observer. Yet, the amygdala does not always show greater response to direct as compared to averted gaze for all emotions, but rather supports its involvement in the appraisal of self-relevance (N'Diaye et al. Reference N'Diaye, Sander and Vuilleumier2009). Brain-imaging and patient studies suggest that the amygdala deals with the integration of emotional expression and gaze direction (Conty et al. Reference Conty, Dezecache and Grèzes2010; Cristinzio et al. Reference Cristinzio, N'Diaye, Seeck, Vuilleumier and Sander2010), with typical differential amygdala activity for anger versus fear processing as a function of direct versus averted gazed (Adams et al. Reference Adams, Pauker and Weisbuch2010; Hadjikhani et al. Reference Hadjikhani, Hoge, Snyder and de Gelder2008; Sato et al. Reference Sato, Kochiyama, Uono and Yoshikawa2010). Therefore, amygdala responses to a given facial expression vary as a function of self-relevance, that is, on concomitant gaze direction and perceived social meaning. Thus, one can advance that embodied simulation can be initiated and/or reinforced either by mutual gaze (as in the case of happiness or anger) or by averted gaze (as in the case of fear).

A possible source of differential self-relevance processing as a function of gaze and emotion is ontogenetic development. Indeed, it has been argued that an early sensitivity to eye contact serves as a major foundation for later development of social skills (Csibra & Gergely Reference Csibra and Gergely2009). In line with this view, the processing relative to averted gaze perception seems to emerge several weeks after the initial sensitivity to eye contact. Only by the age of 3 months do infants show the ability to automatically follow another's eye gaze toward the surrounding space (D'Entremont et al. Reference D'Entremont, Hains and Muir1997). Furthermore, the spontaneous shift of spatial attention involved by another's averted gaze perception at an early age occurs when initial eye contact happened between the infant and the caregiver (Farroni et al. Reference Farroni, Mansfield, Lai and Johnson2003). It is as if the initial attentional capture triggered by eye contact (Farroni et al. Reference Farroni, Csibra, Simion and Johnson2002) was gradually extended to other eye directions (see Reddy [Reference Reddy2003] for similar proposal). Of interest here, the ability to discriminate between emotions only appears to be well established after the second half of the first year (Montague & Walker-Andrews Reference Montague and Walker-Andrews2001) and, once arising, brain processing of emotional faces does integrate the information provided by eye direction (Hoehl & Striano Reference Hoehl and Striano2008). Recognizing various emotional expressions therefore emerge after the ability to process eye direction, whether it is directed toward the self or toward an external object.

It may well be the case that Niedenthal et al. reveal a very basic process by proposing that eye contact increases embodied simulation when perceiving smiling faces. Indeed, the exchanges of smiles and eye contact between a mother/father and her/his infant is probably the most primary form of human social relationship, and could represent the most basic bricks in the development of embodied simulation. However, this phenomenon must evolve over the course of development to understand the diversity of social contents. Thus, the mechanisms primarily involved in eye contact perception may gradually be extended to other eye directions and become a fundamental determinant of emotion recognition. Still, a theoretical question remains: Must eye-gaze direction already somehow be integrated with facial expression before any simulation can be triggered? If integration takes place before simulation, this would mean that one has to recognize, at least very coarsely, the expression before simulating it. As integration certainly arises only after both gaze direction and emotion expression have been processed independently (see Pourtois et al. Reference Pourtois, Sander, Andres, Grandjean, Reveret, Olivier and Vuilleumier2004), we wonder what would trigger simulation in the first place, before it is possibly reinforced by mutual or averted gaze.