2.1.1. Constituents of a second-person approach: Emotional engagement

A second-person grasp of other minds, we argue, is, first, closely related to feelings of engagement with and emotional responses to the other. According to this viewpoint, awareness of other minds crucially hinges upon emotional engagement and a responsiveness to another person's states or actions as compared to a detached observer's attitude, which does not include such responding (cf. Reddy Reference Reddy2003; Reference Reddy2008; Rietveld Reference Rietveld2008) (Fig. 1). Although emotional responses are also influenced by one's own concerns, they constitute an important way of perceiving and integrating the state of the other by way of experiencing one's own bodily responses to her (e.g. Prinz Reference Prinz and Solomon2003) (Fig. 1B). Mentality – in this view – is thought to become meaningful as a result of relating new pieces of experience to a pre-existing network of experiences. On a neural level, emotional and embodied responses may play a crucial role in influencing action control networks of the brain by modulating processes of sensorimotor integration, which, in turn, solicit activity and observable behavior (Frijda Reference Frijda1986, p. 469; Keltner & Haidt Reference Keltner and Haidt1999; Schilbach et al. Reference Schilbach, Eickhoff, Mojzisch and Vogeley2008a) (Fig. 1). Consequently, affective responses relying upon proprioceptive afferences from the body and action-based processes are likely to be closely linked and to interact in complex ways. For instance, it would seem plausible that emotional-embodied responses could facilitate more cognitive ways of understanding minds, whereas the absence of such responses could make this a more effortful process. Furthermore, it is important to point out that while emotional engagement may also occur (and could be particularly relevant) during observation (such as watching an emotionally charged movie scene or perceiving to be perceived by someone; Fig. 1B), the opposite may be true for some situations, in which one actually interacts with someone (such as hastily buying a bus ticket from a cashier; Fig. 1C).

2.1.2. Constituents of a second-person approach: Social interaction

Second, we see social interaction as a key constituent of grasping other minds. Social interactions are characterized by intricate reciprocal relations with the perception of socially relevant information prompting (re-) actions, which are themselves processed and reacted to (see Fig. 1, C & D). These reciprocities might be sequential (as depicted in the schematic of Figure 1), but are often complex, dynamic and nonlinear (Froese & Di Paolo Reference Froese and Di Paolo2010; Port & van Gelder Reference Port and van Gelder1995; Thelen & Smith Reference Thelen and Smith1994). Being in interaction with someone, we suggest, furthermore relies upon a perception of the environment in terms of the resources held collectively by both interactors rather than those held by each individual alone (Costall Reference Costall1995; Marsh et al. Reference Marsh, Richardson and Schmidt2009; Richardson et al. Reference Richardson, Dale and Kirkham2007a; Reference Richardson, Marsh and Baron2007b; Schilbach et al. Reference Schilbach, Eickhoff, Cieslik, Kuzmanovic and Vogeley2012b; Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006). In social interaction, rather than exercising one's deliberative reflective capacities we exercise our own practical know-how in dealing with others as interactors (Klin et al. Reference Klin, Jones, Schultz and Volkmar2003). Here, interaction and feedback are not only a way of gathering data about the other person, that is, observing effects one may have on the other, but rather, as De Jaegher et al. (Reference De Jaegher, Di Paolo and Gallagher2010) have argued persuasively, one's knowledge of the other resides – at least in part – in the interaction dynamics “between” the agents. Thus, taking social interaction seriously suggests that there may not be an absolute epistemic gulf between self and other, which would make an inferential detour necessary, but rather, that the dynamics of the social interaction contribute to and – at times – constitute our awareness of other minds (Fig. 1, C & D).

There are at least three different aspects of social interaction that are important for research in social neuroscience. First, interaction involves different roles for the interactors. At the simplest level, one can be an “initiator” or a “responder” in an interaction, that is, the same (or very similar) action can be performed in an attempt to initiate an interaction or as a response to someone else's action. Although such a simple and clear-cut difference in roles may be rare in dynamic real-time encounters, we believe that this distinction is important for neuroscience research, because it implies different motivational consequences and differences in the underlying neural processes: An action by an “initiator” may involve stronger monitoring of the outcome of the interaction, whereas an action by a “responder” may involve closer attention to the emotional effects of the action being responded to; a difference which may be reflected in differential involvement of brain regions (see Fig. 1). Second, shared intentions and motivations are newly created within an interaction, with important consequences for the performance of joint actions and for the progress and continuation of the interaction itself. We argue that these processes are quite different from those involved in merely observing someone else interact, and must play a crucial role in shaping activity in the neural networks that underlie social cognition (see sects. 3 & 5 for further details). Third, interaction always involves historicity, suggesting that social phenomena must be understood,– at both the psychological and the neural level – within the context of the past and must involve developmental trajectories, which continue to be relevant throughout the life span.

In order to further clarify the central notions of emotional engagement and social interaction as part of a second-person approach to other minds and to illustrate that this view has already begun to prove productive in shaping research outside the neurosciences, in what follows, we draw on and review evidence from the fields of developmental, social, and cognitive psychology.

2.2.1. Developing awareness of minds through second-person engagements: Evidence from developmental psychology

Debates in developmental psychology concerned with the processes through which infants and young children come to recognize the existence and nature of other minds reflect the same assumptions that have beset the traditional philosophical debates. They have most often posited two sources of information for the developing child – that of first-person experience and third-person observation – or, on occasion, an unconvincing amalgam of the two. The possibility of a difference in awareness in different relations with specific “other minds” – which is the central assumption of a second-person approach to mind knowledge (Reddy Reference Reddy2003; Reference Reddy2008) – was either neglected or explicitly denied (Barresi & Moore Reference Barresi and Moore1996). There has, however, been a dramatic surge of interest in second-person explanations within development (e.g., Carpendale & Lewis Reference Carpendale and Lewis2004; Gallagher Reference Gallagher2001; Hobson Reference Hobson1991; Hobson Reference Hobson1999; Reddy Reference Reddy1996; Reference Reddy2003; Reference Reddy2008).

Evidence for the second-person argument, namely that an appropriate development of awareness of other minds depends on the infant first experiencing minds which are directed towards her, is scattered throughout the developmental psychological literature. Probably the most impressive evidence of infants' sensitivity to, and preference for, the infant-directedness of interpersonal actions comes from recent studies concerning early responses to gaze. Mutual gaze both reflects one organism's interest in another's gaze and signals this interest to the other, with the potential for creating a situation of intense mutuality. Between 2 and 5 days of birth, human neonates show a preference for looking at faces or pictures of faces with eyes directly looking towards the infant. This preference reveals itself both in terms of longer looking times and in terms of more frequent looks (Farroni et al. Reference Farroni, Csibra, Johnson and Simion2002). Even earlier than this, within minutes of birth, infants show considerable interest in and respond appropriately to self-directed facial actions, primarily more noticeable actions such as tongue protrusion and opening the mouth wide (Kugiumutzakis Reference Kugiumutzakis and Braten1998; Meltzoff & Moore Reference Meltzoff and Moore1977; Nagy & Molnar Reference Nagy and Molnar2004). Also, there is evidence that by 4 months of age the experimenter's gaze directed toward rather than away from the infant leads to a variety of neural reactions and information processing advantages: higher cortical arousal (Farroni et al. Reference Farroni, Csibra, Johnson and Simion2002), enhanced occipital early gamma response (Grossmann et al. Reference Grossmann, Johnson, Farroni and Csibra2007), enhanced neural processing of emotional expressions (Striano et al. Reference Striano, Kopp, Grossman and Reid2006), facilitated neural processing of objects (Reid et al. Reference Reid, Striano, Kaufman and Johnson2004), and differential processing of words (Parise et al. Reference Parise, Palumbo, Handl and Frederici2008). By 2 months of age infants already show a variety of emotional reactions to attention directed to the self, ranging from pleasure (Wolff Reference Wolff1987) to distress if unable to disengage (Brazelton Reference Brazelton, Faulkner and Tanner1986) to ambivalence in the form of coy smiles (Reddy Reference Reddy2000). A large number of studies have shown that the intricacies of emotional engagement matter from very early infancy and are the source of meaning about self and others. Two- to three-month-olds during face-to-face communication demonstrate intra- and interpersonal coordination (Fogel Reference Fogel1993; Lavelli & Fogel Reference Lavelli and Fogel2002; Stern Reference Stern1985; Trevarthen Reference Trevarthen and Schaffer1977), are disturbed by the lack of temporal and affective coordination (Cohn & Tronick Reference Cohn and Tronick1989; Legerstee & Varghese Reference Legerstee and Varghese2001; Murray & Trevarthen Reference Murray, Trevarthen, Field and Fox1985; Nadel & Tremblay-Leveau Reference Nadel, Tremblay-Leveau and Rochat1999), are affected in their ability to bid for attention by the history of maternal affective responses (McQuaid et al. Reference McQuaid, Bibok and Carpendale2009) and learn to prefer the levels of interactional contingency they are familiar with (Bigelow & Rochat Reference Bigelow and Rochat2006).

Infant awareness of attention in second-person engagements has been argued to have developmental primacy over the awareness of attention in triadic and more temporally extended attentional engagements (Reddy Reference Reddy2003; Reference Reddy, Eilan, Hoerl, McCormack and Roessler2005; Reference Reddy2008). Evidence for the causal role of such engagements in developing triadic and more complex reflective awareness comes from developmental psychopathology (particularly autism) where problems with explicit social cognition (at the level of triadic joint attention or at the level of concepts of self or of other) are heralded by problems in second-person engagements (with establishing mutual attention and mutual responsiveness with self and with other; Leekam & Ramsden Reference Leekam and Ramsden2006; Reddy, Reference Reddy and Seemann2012; Reddy et al. Reference Reddy, Williams, Costantini and Lang2010).

The primacy of second-person engagements creates serious conceptual and methodological problems for psychological research: It demands that emotion be taken as central to an awareness of minds and focuses on emotional responses rather than reflections or constructs. We argue that such experiences of mind and mentality are first and most intensely experienced within second-person engagements, where the individual is directly addressed by and responds to an “other” mind. Such second-person experiences of other minds, we argue, are necessary for the typical development of social understanding and continue to influence social understanding throughout life.

2.2.2. Knowing minds in interaction: Evidence from social and cognitive psychology

In social psychology and related fields – in spite of a long history and interest in “social presence” effects (e.g., Allport Reference Allport1924; Biocca et al. Reference Biocca, Harms and Burgoon2003; Short et al. Reference Short, Williams and Christie1976; Triplett Reference Triplett1898; Worringham & Messick Reference Worringham and Messick1983; Zajonc Reference Zajonc1965) – the need to allow test subjects to leave the “experimental quarantine” of “isolation paradigms” when studying social processes has only recently been recognized (e.g., Becchio et al. Reference Becchio, Sartori and Castiello2010; Richardson et al. Reference Richardson, Dale and Kirkham2007a; Reference Richardson, Marsh and Baron2007b; Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006). This development appears to be based on the assumption that cognition is grounded in basic perception and action processes and emerges out of the interaction of the organism with its environment, and that – rather than treating it as an experimental confound – a social context and social interaction can be treated as an independent variable of experimentation. Consistent with this proposal, the second-person approach sees interaction with other agents and the ensuing interaction dynamics as crucial for the knowing of other minds and the expression of that knowledge.

The role of social interaction for cognitive and social development has begun to gain center stage in various scientific discourses (De Jaegher et al. Reference De Jaegher, Di Paolo and Gallagher2010): The role of interaction as a vehicle for the acquisition of knowledge has, for instance, been demonstrated in language development. In contrast to the Chomskian idea of a “Language Acquisition Device” (e.g., Chomsky Reference Chomsky1979), the perception of structure in social interactions has been shown to guide vocal development and language learning (Bruner Reference Bruner1983; Goldstein & Schwade Reference Goldstein, Schwade, Blumberg, Freeman and Robinson2010), both in terms of speech perception (Kuhl Reference Kuhl2007a; Reference Kuhl2007b; Kuhl et al. Reference Kuhl, Tsao and Liu2003) or turn-taking (Goldstein & Schwade Reference Goldstein and Schwade2008; Goldstein et al. Reference Goldstein, Schwade, Briesch and Syal2010a; Jaffe et al. Reference Jaffe, Beebe, Feldstein, Crown and Jasnow2001). Whereas children can extract statistical regularities from any structured material without awareness or intention using mere error feedback (Marcus et al. Reference Marcus, Vijayan, Rao and Vishton1999; Saffran et al. Reference Saffran, Newport, Aslin, Tunick and Barrueco1997), observation of one's own behavioral outcomes in interaction may be crucial for such learning to occur (Goldstein et al. Reference Goldstein, Waterfall, Lotem, Halpern, Schwade, Onnis and Edelman2010b). Adults also coordinate turn-taking, implicitly agree upon names for objects (Brennan & Clark Reference Brennan and Clark1996; Brennan & Hanna Reference Brennan and Hanna2009), approach each other's accents (Giles et al. Reference Giles, Coupland, Coupland, Giles, Coupland and Coupland1992), sway their bodies in synchrony (Shockley et al. Reference Shockley, Santana and Fowler2003), and visually coordinate their attention through synchronized eye movements (Richardson et al. Reference Richardson, Dale and Kirkham2007a), suggesting that communication may be grounded by a shared context, which is actively developed in ongoing encounters (Clark Reference Clark1996; Garrod & Pickering Reference Garrod and Pickering2004).

In other areas of research “interaction” has been investigated by focusing on processes such as involuntary mimicry (e.g., Chartrand & Bargh Reference Chartrand and Bargh1999; Konvalinka et al. Reference Konvalinka, Vuust, Roepstorff and Frith2010; Niedenthal et al. Reference Niedenthal, Mermillod, Maringer and Hess2010; Schmidt et al. Reference Schmidt, Carello and Turvey1990), which leads to enhanced rapport and liking, but is also influenced by differences in affiliative motives and interdependent self-construal (Van Baaren et al. Reference Van Baaren, Maddux, Chartrand, De Bouter and Van Knippenberg2003; Baaren et al. Reference Van Baaren, Holland, Kawakami and van Knippenberg2004). Social interaction, however, normally also involves more complex forms of coordination present from early on in life (cf. Harrist & Waugh Reference Harrist and Waugh2002). In this respect, Knoblich & Sebanz (Reference Knoblich and Sebanz2008) distinguish between “action simulation” (see also Rizzolatti & Sinigaglia Reference Rizzolatti and Sinigaglia2010), “joint attention,” and “shared intentionality” (see also Tomasello & Carpenter Reference Tomasello and Carpenter2007). The development of a shared perceptual and attentional space within early mutual attentional engagements is crucial, because it paves the way for triadic interactions in which we share our mental states about a third object or person with the other (Tomasello Reference Tomasello, Moore and Dunham1995). In turn, such triadic attentional engagements may be crucial for developing complex joint actions (Fiebich & Gallagher Reference Fiebich and Gallagher2012; Knoblich & Sebanz Reference Knoblich and Sebanz2008; Richardson et al. Reference Richardson, Dale and Kirkham2007a; Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006).

In spite of a growing interest in this matter, the bulk of research is still rooted in a mechanistic metaphor of feedback loops that inform individuals about the adequacy of their behavior. In this framework, it is assumed that the sum of two interacting agents can be expressed in terms of processes “within” those agents (however, modified by the external world, in this case each other). We argue that interaction expresses a type of knowledge of other minds that is not entirely reducible to knowledge “inside” any two individual agents but exists “between” them (see De Jaegher et al. Reference De Jaegher, Di Paolo and Gallagher2010). Interestingly, the recently developed “perceptual crossing paradigm” directly addresses this aspect (Auvray et al. Reference Auvray, Lenay and Stewart2009; Lenay et al. Reference Lenay, Stewart, Rohde and Ali Amar2011): here, two participants interact with each other by moving their mouse cursor in a one-dimensional virtual space. In this space they can encounter three different types of objects indicated by tactile stimulation: a fixed object, a constantly moving object, and the other participant. Their task is to click the mouse only if they believe to have encountered the other person's cursor. Thus, it is only when they meet each other, that both are stimulated simultaneously. Using the perceptual crossing setup, it has been shown that healthy volunteers exhibit a dissociation between task performance and awareness of this performance with respect to the detection of reciprocity (though not with respect to the detection of animacy): Conscious identification of the other person (by means of clicks) is limited to distinguishing both moving objects from the fixed object (animacy detection), but does not extend to distinguishing the moving object from the other person's object (reciprocity detection). However, participants' interaction dynamics allow us to distinguish not only between fixed and moving objects, but also between the moving object and the other's cursor (Auvray et al. Reference Auvray, Lenay and Stewart2009). Crucially, this demonstrates that unrelated to awareness and strategic processes, people are sensitive to reciprocal interaction, and furthermore, spontaneously engage in mutual modification of action patterns with an interactor, actively “seeking the other out.” It is only in this interactive process that contingencies are simultaneously experienced and acted upon.

Taken together, recent progress in developmental, social, and cognitive psychology indicates a surge of interest in emotional engagement and real-time social interaction as key constituents for knowing other minds. However, although this second-person approach has already begun to prove productive in shaping experimental work, this has occurred largely outside the realm of neuroscience. We suggest that the approach promises to do the same within the field of social neuroscience, thereby leading to the establishment of a second-person neuroscience.

3.1.1. Being addressed as you: The neurobiological correlates of emotional engagement

Based on the hypothesis of differences in the neural processing of social stimuli depending upon whether or not they are directed toward oneself as compared to being directed toward someone else, we performed a functional magnetic resonance imaging (fMRI) study (Schilbach et al. Reference Schilbach, Wohlschlaeger, Kraemer, Newen, Shah, Fink and Vogeley2006), in which participants were asked to imagine being part of a scenario with three other people, only one of whom was visible (Fig. 3A), and were shown short video sequences depicting this virtual character who exhibited socially relevant facial expressions or arbitrary facial movements. Furthermore, it was varied whether or not the virtual character looked directly at the participant (Fig. 3B) or towards the right or left in the direction of another agent (Fig. 3C). Participants were asked to rate in all cases whether they felt that the virtual character had expressed an intention to initiate a social interaction.

Figure 3.

(A) Virtual scenario as shown in the instructions. (B) Self-directed, socially relevant facial expression. (C) Other-directed, arbitrary facial movement. (D) Neural correlates of the perception of self-directed facial expressions. (E) Neural correlates of the perception of other-directed facial expressions. (F) Neural correlates of the perception of arbitrary facial movements. Activation overlays based on reanalysis of data taken from Schilbach et al. (Reference Schilbach, Wohlschlaeger, Kraemer, Newen, Shah, Fink and Vogeley2006).

The behavioral results demonstrated that participants did not have difficulties in differentiating socially relevant from arbitrary mimic behavior. Interestingly, and in spite of the explicit instruction to “put themselves into the shoes of the other,” participants demonstrated a bias towards giving socially relevant facial expressions a significantly higher rating when they were self-directed. In parallel to this, results of the neuroimaging analysis demonstrated significant differences in activation patterns subserving the perception of socially relevant facial expressions, depending upon whether these expressions were self- or other-directed: While self-directed facial expressions led to a differential increase of neural activity in the ventral portion of medial prefrontal cortex (MPFC) and the (superficial) amygdala (AMY) (Fig. 3D), other-directed facial expressions resulted in a differential recruitment of medial and lateral parietal cortex (Fig. 3E). Our findings, therefore, demonstrate that while the perception of socially relevant nonverbal cues from an observer's point of view recruits brain regions that have been related to visuo-spatial processing, the perception of identical, self-directed stimuli activate brain regions that have been related to emotional and evaluative processing (Schilbach et al. Reference Schilbach, Wohlschlaeger, Kraemer, Newen, Shah, Fink and Vogeley2006).

Indeed, the AMY is known to be a crucial subcortical hub of the limbic system serving various integrative sensory and emotional functions in attaching biological and social significance to stimuli (Gamer & Buchel Reference Gamer and Buchel2009; Gamer et al. Reference Gamer, Zurowski and Buchel2010; Muscatell et al. Reference Muscatell, Addis and Kensinger2010; N'Diaye et al. Reference N'Diaye, Sander and Vuilleumier2009; Tamietto & de Gelder Reference Tamietto and de Gelder2010). The ventral portion of MPFC is also known to be relevant for emotional processing and has been implicated to be involved in “outcome monitoring” (Amodio & Frith Reference Amodio and Frith2006). More generally, emotions are known to be an important driving force for interpersonal behavior and a “short hand” that guides decision-making processes (e.g., Damasio Reference Damasio2010; Frijda Reference Frijda1986). Also, emotions are assumed to be of crucial importance for interpersonal perception by allowing the observer to share the feeling state of another individual (Preston & de Waal Reference Preston and de Waal2002). Apart from this traditional view of how emotions might contribute to social cognition, our account views emotional responses not only as a way of perceiving emotional states in others, but also as a way of being engaged with others, which could contribute to the understanding of the bodily intentionality of the other in terms of bodily responsiveness (Rietveld Reference Rietveld2008). Being responsive to the socially relevant, expressive behavior of others in this way could provide, and bias, possibilities for (inter-) action by motivating behavior and soliciting activity.

Interestingly, our neuroimaging analysis also demonstrated that the perception of arbitrary facial movements recruited the right inferior frontal and inferior parietal cortices (see Fig. 1 & Fig. 3F). These brain regions are commonly described as important nodes of the mirror neuron system (MNS), which has been related to understanding the (motor) intentions that underlie others' actions (Rizzolatti & Sinigaglia Reference Rizzolatti and Sinigaglia2010). This finding, we think, is interesting in several respects: First of all, it suggests that activity in the MNS is not related to the particular perspective a human observer adopts, but more closely tracks the observed behavior itself regardless of whether it is “my action” or “your action.” This seems to be consistent with the dominant interpretation of MNS activity in the literature, suggesting that the MNS may have evolved to represent actions in the same way across subjects and that mirror neurons explain the ability to understand other minds by giving us a “first-person grasp” of the motor goals and intentions of other individuals (Rizzolatti & Sinigaglia Reference Rizzolatti and Sinigaglia2010). This view has, however, been criticized as over-emphasizing similarities in self- and other-related processes (Schilbach Reference Schilbach2010) and under-representing the important question of where mirror neurons actually “come from” (Heyes Reference Heyes2010). Here, an important suggestion has been that mirror neurons may only play a role in situations when the action and its stereotypic context are highly familiar (e.g., Brass et al. Reference Brass, Schmitt, Spengler and Gergely2007). In our study, however, brain regions described as corresponding to the MNS respond more strongly to arbitrary facial movements than to socially relevant ones – which appears to contradict the idea that the MNS only plays a role in familiar situations. On the contrary, an arbitrary facial movement – with which one may be less familiar than with socially relevant facial expressions which are directed towards another person more frequently – also leads to the recruitment of the MNS. This finding could be understood in terms of a prediction error signal (Pelphrey et al. Reference Pelphrey, Singerman, Allison and McCarthy2003; Schippers & Keysers Reference Schippers and Keysers2011) and could be relevant to the associative learning processes recently discussed as an alternative account of the development and modulation of activity in the MNS (Heyes Reference Heyes2010).

In fact, according to the “associative hypothesis” (Heyes Reference Heyes2010), mirror neurons are forged through sensorimotor experience of observing and executing actions, and the MNS, even in adulthood, can be reconfigured through sensorimotor learning. Evidence for this comes from a series of elegant studies which demonstrate that sensorimotor experience can enhance (Press et al. Reference Press, Gillmeister and Heyes2007), abolish (Heyes et al. Reference Heyes, Bird, Johnson and Haggard2005), and even reverse (Catmur et al. Reference Catmur, Walsh and Heyes2007; Reference Catmur, Gillmeister, Bird, Liepelt, Brass and Heyes2008) “mirror” activation in human participants. Consistent with this finding, a recent simulation study demonstrated that a reinforcement learning based model could account for the development of mirror neurons in gaze following (Triesch et al. Reference Triesch, Jasso and Deak2007). With respect to the possible sources of sensorimotor experience that may impact on the MNS, Heyes (Reference Heyes2010) has recently emphasized that much of this experience is obtained through interaction with others. We strongly support this position and argue that the investigation of action observation and execution in a more ecologically valid, interpersonal context – as suggested by a second-person neuroscience – will help to investigate these matters further (cf. Kourtis et al. Reference Kourtis, Sebanz and Knoblich2010). More specifically, we see a role for the MNS beyond mere motor resonance: Rather than suggesting that mirror neuron activity constitutes a simulation routine which gives us a “first-person grasp” of the other (Rizzolatti & Sinigaglia Reference Rizzolatti and Sinigaglia2010), we contend that an understanding of the MNS on the enactive view of cognition would suggest that activity in these brain regions might be more closely related to the ability to perceive social affordances. Such affordances can thereby be understood as the possibilities for interaction provided by others, in terms of an activation of motor programs that would allow for interpersonal coordination of behavior (cf. Gangopadhyay & Schilbach, Reference Gangopadhyay and Schilbach2012). This view is consistent with the idea of social interactions having a profound impact on MNS activity and offers a more plausible account for actions that people perform jointly, as these often involve complementary or incongruent actions rather than imitative or congruent ones (cf. Fuchs & De Jaegher Reference Fuchs and De Jaegher2009; Gallagher Reference Gallagher2007). Indeed, a study by Newman-Norlund et al. (Reference Newman-Norlund, van Schie, van Zuijlen and Bekkering2007) found evidence that the MNS is more active during the preparation of complementary actions, while other studies demonstrate that activity in the human MNS can be modulated by social interaction (Kourtis et al. Reference Kourtis, Sebanz and Knoblich2010; Oberman et al. Reference Oberman, Pineda and Ramachandran2007). This is clearly in line with our suggestion that an essential aspect of apprehending other minds lies in the kinds of responses their activity elicits (see Fig. 1).

In a follow-up experiment to our fMRI study, the paradigm was used while recording eye-movements, pupil size, and facial electromyography (EMG). The results show that attention allocation, as assessed by fixation duration, was specifically related to the perception of self-directed stimuli. EMG measurements demonstrated that facial activity was influenced by the perception of socially relevant facial expressions, demonstrating spontaneous, involuntary facial responses that have been described as “facial mimicry” (Mojzisch et al. Reference Mojzisch, Schilbach, Helmert, Pannasch, Velichkovsky and Vogeley2006). Although we did not observe a statistically significant interaction effect here, the difference in EMG activity between self- and other-directed facial expressions was noteworthy (see Mojzisch et al. Reference Mojzisch, Schilbach, Helmert, Pannasch, Velichkovsky and Vogeley2006; Schilbach et al. Reference Schilbach, Eickhoff, Mojzisch and Vogeley2008a for details). Using the temporal information from the EMG study, we re-analyzed the fMRI data to investigate the neural correlates of facial mimicry. This analysis identified a network comprising the face region of the primary motor cortex, but also the posterior cingulate cortex, hippocampus, and the dorsal midbrain. This is suggestive of a close link between the perception of facial expressions and their generation in the observer. Facial expressions might be particularly prone to engaging such mechanisms as they serve important communicative functions. Furthermore, these findings demonstrate that situations which lead to emotional engagement and invite an element of a motor response, recruit both fronto-parietal areas, as well as the cortical midline structures, and differentially activate the limbic system of the brain (Schilbach et al. Reference Schilbach, Eickhoff, Mojzisch and Vogeley2008a; cf. Buck et al. Reference Buck, Losow, Murphy and Constanzo1992; see Fig. 1).

Taken together, the results of these studies demonstrate significant differences in the neurobiological mechanisms of the perception of facial expressions depending upon emotional engagement. When we are personally addressed by others, the perception of their mimic behavior relies, in neurobiological terms, upon tight perception-action coupling with affective and body-based processing feeding into and promoting the preparation of motor responses as a way of picking up and responding to the possibilities for interaction.

3.1.2. Minds made for sharing: The neurobiological correlates of the reciprocity of social interaction

As outlined above, interaction loops must be considered as important constituents of second-person relations. Thus, within social interaction, interactors mutually and directly influence each other and may hold different roles during the interaction (as introduced in sect. 2.1; see Fig. 1, C & D). To establish a paradigm that allows participants to be part of a social interaction in the constrained fMRI setting, we have developed a novel technique that makes use of eyetracking data obtained from participants inside the scanner to control a virtual character's gaze behavior in real-time making it “responsive” to the human observer's gaze (Wilms et al. Reference Wilms, Schilbach, Pfeiffer, Bente, Fink and Vogeley2010). This setup, therefore, allows participants to experience their own eye-movements to have an effect on the gaze behavior of a (virtual) other, similar to how this occurs in real-life situations. Importantly, such a setup allows for the investigation of cognitive phenomena which are at the interface of automatic and controlled processes, because eye-movements are known to be closely related to both (e.g., Hikosaka & Isoda Reference Hikosaka and Isoda2010). Also, the setup allows us to investigate phenomena whose emergence necessarily depends upon social interaction, its reciprocity; and whose underlying psychological processes and neural mechanisms may differ depending upon the roles interactors hold during interaction, namely those of being “initiator” or “responder” in the interaction.

With respect to the phenomenon of joint attention, that is, attending to something together with someone and being aware that “we both” are attending, the difference between being “initiator” or “responder” in the interaction appears to be highly relevant: Joint attention can occur either as a result of following someone else's gaze toward an object (“responder”) or as a result of directing someone else's gaze toward the object (“initiator”). Clinically, children with autism, in fact, show less pronounced impairments in their ability to respond to a person's gaze shifts (by following the gaze) than in their spontaneous drive to initiate the sharing of attention (by trying to make someone look at something; Mundy & Newell Reference Mundy and Newell2007).

Based on the hypothesis that differences in the neural basis of joint attention could be related to the reciprocity of social interaction (as compared to previous social neuroscience attempts to investigate the neural correlates of joint attention by means of tasks in which participants are mere observers of gaze cues (e.g., Materna et al. Reference Materna, Dicke and Thier2008; Williams et al. Reference Williams, Waiter, Perra, Perrett and Whiten2005; see also sect. 2.1), the setup described above was used to perform an fMRI study, in which participants interacted with the virtual other while undergoing neuroimaging (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b). Experimental variations consisted of initiating versus following the gaze of the virtual character when fixating objects shown on the stimulus screen (Fig. 4A). Our results demonstrate the recruitment of medial prefrontal cortex (MPFC) and posterior cingulate cortex (PCC) for joint attention to an object, regardless of whether or not this was initiated by the participant or the (virtual) other (Fig. 4B). Conversely, looking at an object different from the one inspected by the virtual other led to the recruitment of a lateralized fronto-parietal network (Fig. 4C). While the latter closely resembles the neural network which has been related to the control of attention and eye-movements (Corbetta et al. Reference Corbetta, Patel and Shulman2008; Pierrot-Deseilligny et al. Reference Pierrot-Deseilligny, Milea and Muri2004), the former regions closely match those which have been described as belonging to the “mentalizing network” (MENT) (Frith & Frith Reference Frith and Frith2006; Kampe et al. Reference Kampe, Frith and Frith2003), overlapping with others that have become known as the “default mode of brain function” (Raichle et al. Reference Raichle, MacLeod, Snyder, Powers, Gusnard and Shulman2001; Schilbach et al. Reference Schilbach, Eickhoff, Rotarska-Jagiela, Fink and Vogeley2008b; Schilbach et al. Reference Schilbach, Bzdok, Timmermans, Fox, Laird, Vogeley and Eickhoff2012a).

Figure 4.

(A) Screenshot depicting an anthropomorphic virtual character and three objects (grey squares). (B) Neural correlates of main effect of joint attention. (C) Neural correlates of non-joint attention. (D) Neural correlates of other-initiated joint attention. (E) Neural correlates of self-initiated joint attention. Activation overlays based on a reanalysis of data taken from: Schilbach et al. (Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b).

In spite of participants always fixating on an object on the stimulus screen, the underlying brain activity appears to be markedly different depending upon whether or not the participant is doing this “together” with a virtual other. Doing so in coordination with the virtual character resulted in a differential increase of neural activity in a network which has been related to the human ability for grasping another person's mental states and her communicative intentions, and for using an internally oriented mode of cognizing to contextualize aspects of a situation (Amodio & Frith Reference Amodio and Frith2006; Bar Reference Bar2007; Buckner & Carroll Reference Buckner and Carroll2007; Kampe et al. Reference Kampe, Frith and Frith2003; Schilbach et al. Reference Schilbach, Eickhoff, Rotarska-Jagiela, Fink and Vogeley2008b). Here, it is important to stress that the MENT was activated as a result of participants simply and naturally coordinating their gaze with that of the other without any explicit instruction to assess the mental states of the virtual other. This is consistent with the observation of activations of dorsal MPFC when study participants were addressed by their first names or by faces looking directly at them (Kampe et al. Reference Kampe, Frith and Frith2003). Our findings, therefore, seem to contradict the proposal that the MENT is responsible for reflective and more explicit social cognition (Keysers & Gazzola Reference Keysers and Gazzola2007). In essence, we are able to show that this network is activated as a result of interpersonal coordination in social interaction, which was also experienced as more pleasant and less effortful than doing the opposite of what the virtual character was doing (cf. Schnell et al. Reference Schnell, Bluschke, Konradt and Walter2011). Follow-up studies using modifications of our novel paradigm will investigate putative connectivity differences between areas of the MENT during social interaction and observation, and will explore how a cooperative or competitive context influences the neural mechanisms of a gaze-based interaction (Fig. 1; see sects. 3.2 and 5 for further details).

Our findings, therefore, raise the intriguing possibility that activity changes in the MENT can occur as a result of (gaze-based) social interaction (cf. Allen & Williams Reference Allen and Williams2011). To us it is tempting to think of this finding as paralleling the suggestions of Heyes (Reference Heyes2010) with respect to the MNS, giving rise to the hypothesis that similar associative mechanisms might play a role in the case of the MENT. In a similar line of thought, Behrens and colleagues have demonstrated that social prediction error signals – when learning from the behavior of or advice provided by others – are coded by MENT components, which they describe as evidence for the argument that higher-order social cognitive mechanisms may rely on simple associative processes (Behrens et al. Reference Behrens, Hunt and Rushworth2009). Crucially, the elegant computational approach taken by Behrens et al. to investigate reward-based learning and decision-making uses a generative model, allowing the consideration of the learning history to predict changes in neural activity over time. Similarly, we suggest, future research in social neuroscience should investigate the development and changes in neural networks, that is, the neural plasticity related to and induced by real-time social interactions, thereby helping to understand how particular functions actually develop in particular areas of the brain or in brains of interactors (cf. Westermann et al. Reference Westermann, Mareschal, Johnson, Sirois, Spratling and Thomas2007; see Fig. 1 and sect. 5 for more details).

Furthermore, our neuroimaging finding of a differential increase of neural activity in the MENT is particularly relevant, because it is observed during joint attention: Ontogenetically, joint attention is known to be a most important precursor of explicit social cognitive abilities and predates these conceptual abilities by years: A recent study was, indeed, able to show that children have an implicit “know how” of what others experience when they encounter them in joint engagement, long before passing false-belief tasks (Moll et al. Reference Moll, Carpenter and Tomasello2007). Based on our findings, it can be assumed that joint attention in social interaction already relies upon the brain regions of the MENT before children pass tasks that require an explicit knowledge of others' mental states (cf. Grossmann & Johnson Reference Grossmann and Johnson2010). Rather than being a network that is exclusively devoted to reflective social cognition, this network might, therefore, be trained up by participation in social interaction in terms of basic mechanisms which allow the use of analogies and associations to generate predictions (Friston Reference Friston2008; Mitchell Reference Mitchell2009). During later stages in development this network may then be brought into the service of explicit social cognition (Frith & Frith Reference Frith and Frith2008; Knops et al. Reference Knops, Thirion, Hubbard, Michel and Dehaene2009), which – at the level of the brain – could be reflected in differences in the functional or effective connectivity between individual brain regions, rather than functionally segregated effects (e.g., Daunizeau et al. Reference Daunizeau, den Ouden, Pessiglione, Kiebel, Friston and Stephan2010a; Reference Daunizeau, den Ouden, Pessiglione, Kiebel, Stephan and Friston2010b; Sporns et al. Reference Sporns, Chialvo, Kaiser and Hilgetag2004). In these terms, the progression from perceiving and being motivated to respond to relevant social stimuli to understanding them may be a result of the matching of top-down and bottom-up expectations. Importantly, bi-directional relationships between different levels of neural processing may remain relevant throughout life (Teufel et al. Reference Teufel, Fletcher and Davis2010; see Fig. 1).

Returning to the results of the fMRI study on joint attention, it is of crucial importance that differences in the neural correlates were also observed depending upon joint attention being self- or other-initiated: While following someone else's gaze to engage in joint attention resulted in the recruitment of the anterior portion of MPFC (aMPFC; Fig. 4D), known to be involved in the enhancement of stimulus-oriented attending (Burgess et al. Reference Burgess, Dumontheil and Gilbert2007), directing someone else's gaze towards an object activated the ventral striatum (VS), a part of the functional neuroanatomy of reward processing (Fig. 4E). In light of ratings of subjective experience also obtained from participants during a post-scan questionnaire – which indicated that they enjoyed looking at objects more “together with” the virtual other – the latter finding, indeed, appeared to be related to the hedonic aspects of experiencing self-initiated joint attention (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b). This seems to be in line with the idea that social interaction may involve collaborative and affiliative motives, the realization of which is experienced as rewarding, possibly relying upon an “intrinsic” motivation of humans for sharing of experiences (Tomasello Reference Tomasello2009; Trevarthen Reference Trevarthen and Olson1980). Furthermore, these findings demonstrate how basic motivation- or reward-related signals may play a key role in the establishment and maintenance of social relations (e.g., Panksepp et al. Reference Panksepp, Herman, Vilberg, Bishop and DeEskinazi1980), which appears to be a crucial area for future research in social neuroscience (see sect.5.2 for further details). Most importantly, our results provide clear evidence that the neural circuitry used during social interaction differs from the circuitry recruited during “individual” social cognition, when one's behavior does not have effects on the “object” of one's cognition (cf. Materna et al. Reference Materna, Dicke and Thier2008; Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b; Williams et al. Reference Williams, Waiter, Perra, Perrett and Whiten2005).

3.2.1. Top-down and bottom-up investigations of social interaction

In the joint attention paradigm, participants interact with anthropomorphic virtual characters, who they believe are controlled by other human participants (Wilms et al. Reference Wilms, Schilbach, Pfeiffer, Bente, Fink and Vogeley2010). In this paradigm, we are in a position to investigate the behavioral and neural correlates of interaction contingencies and responses that are characteristic of various forms of social encounters (see Figs. 1 & 2). Future studies will make use of this setup to explore the interaction of explicit and implicit processes in social cognition and how this may depend upon being able to “get into the action.” Here, one could specifically ask whether the difference between social cognition from an interactor's as compared to an observer's point of view merely adds an action component or whether this actually changes other psychological processes. In terms of the underlying neural substrates such differences might be characterizable in terms of overlapping versus distinct neurocircuits or could be related to differences in connectivity between “social brain” regions (Fig. 1). Also, the question could be addressed whether ostensive cues exchanged between interactors (e.g., mutual gaze; Csibra & Gergely Reference Csibra and Gergely2009) are equally effective from an observer's point of view as compared to situations when one is personally addressed.

Furthermore, paradigms are being developed in our laboratory which will help to understand differences between social cognition from an interactor's as compared to an observer's point of view with respect to reward-based interactions (cf. Pessoa & Engelmann Reference Pessoa and Engelmann2010). Based on the assumption that social interaction per se may be experienced as motivating and rewarding, it appears plausible to devise paradigms that implement a social context, in which task performance may be modulated depending upon whether or not the motivation to interact socially is parallelized with or contrasted with performing the task (see sect. 5.1.1 for further details). Another extension of the above described joint attention paradigm could consist in using virtual characters whose facial expressions and whole-body movements could also be manipulated. This setup would allow the investigation of how participant-directed gaze behavior of the virtual character would influence the perception of, for example, reaching or grasping movements in peri- and extrapersonal space (cf. Caggiano et al. Reference Caggiano, Fogassi, Rizzolatti, Thier and Casile2009). Here, an interesting possibility would be to parametrically vary the degree to which a virtual character is responsive to a human observer's gaze in real time to investigate how this may lead to differential recruitment of the MNS and MENT of the brain (Brass et al. Reference Brass, Schmitt, Spengler and Gergely2007; de Lange et al. Reference de Lange, Spronk, Willems, Toni and Bekkering2008; Santos et al. Reference Santos, Kuzmanovic, David, Rotarska-Jagiela, Eickhoff, Shah, Fink, Bente and Vogeley2010; Wheatley et al. Reference Wheatley, Milleville and Martin2007). Also, the presentation of real-world objects that can be attended to both by the virtual character and the participant could be helpful to manipulate neural activity in those large-scale neural networks.

Another fundamental question for a second-person neuroscience – closely related to the “problem” of other minds – consists in asking which factors actually determine that we perceive an interaction as an exchange, with a mindful rather than a machine-driven agent. To this end, a “nonverbal Turing test” can be devised, in which participants engage in gaze-based interaction with an anthropomorphic virtual character, whose gaze behavior is either controlled by another participant or by a computer (Fig. 2; Pfeiffer et al. Reference Pfeiffer, Timmermans, Bente, Vogeley and Schilbach2011). These investigations are likely to be informative with respect to the impact of valence (whether or not an interaction subjectively feels positive or rewarding) as compared to contingency (the degree to which the virtual character objectively follows or does not follow one's gaze). Furthermore, one can address the question of the congruency of the behavior shown by the virtual character during the interaction – that is, whether following or not following the participant's gaze has an impact on ratings of sympathy and trustworthiness, as well as subsequently evoked prosocial behavior shown towards the person one has interacted with.

3.2.2. From interaction loops to interaction dynamics

In the early years of social neuroscience, attempts had already been made to investigate two brains in interaction through “hyperscanning,” which was hailed as a break-through technology (Montague et al. Reference Montague, Berns, Cohen, McClure, Pagnoni, Dhamala, Wiest, Karpov, King, Apple and Fischer2002). Although the application of this method has, indeed, provided invaluable insights into the neural basis of social cognition in conditions of health and pathology (e.g., King-Casas et al. Reference King-Casas, Tomlin, Anen, Camerer, Quartz and Montague2005; Reference King-Casas, Sharp, Lomax-Bream, Lohrenz, Fonagy and Montague2008), the approach never really caught on. At least in part this is due to the fact that using it to its full potential would have required establishing more ecologically valid ways for two or more participants to interact (cf. Redcay et al. Reference Redcay, Dodell-Feder, Pearrow, Mavros, Kleiner, Gabrieli and Saxe2010). Most hyperscanning studies, however, have relied upon experimental paradigms from game theory, which allow for a highly structured, tit-for-tat kind of interaction. The latter can be elegantly formalized mathematically, but can also be criticized for not recreating the interaction dynamics of everyday-life social encounters (Engemann et al. Reference Engemann, Bzdok, Eickhoff, Vogeley and Schilbach2012). Also, paradigms from game theory are deeply entrenched in a cognitivist view of the subject as a (rather) rational and (rather) passive recipient of information, which may not fully capture many aspects of naturalistic social interaction, including the ways in which interactors co-regulate their behavior (cf. Bacharach Reference Bacharach2006; Hertwig & Erev Reference Hertwig and Erev2009; Schonberg et al. Reference Schonberg, Fox and Poldrack2011).

However, first steps are now being taken to investigate the neural mechanisms of interaction dynamics (Fig. 2: cubes [4] & [5]). Similar to our approach of using gaze behavior to allow for direct social interaction (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b; Wilms et al. Reference Wilms, Schilbach, Pfeiffer, Bente, Fink and Vogeley2010), the recent work of Saito et al. (Reference Saito, Tanabe, Izuma, Hayashi, Morito, Komeda, Uchiyama, Kosaka, Okazawa, Fujibayashi and Sadato2010) capitalizes on the idea that inter-subject synchronization at the neural level might occur during eye contact. Using a similarly minimalist and yet naturalistic setup allows the experimental control of bandwidth, that is, the degree of complexity of the interaction, which may be helpful in focusing on its temporal dynamics (cf. Auvray et al. Reference Auvray, Lenay and Stewart2009). For their ground-breaking study Saito et al. have devised a setup in which they not only use hyperscanning, but also allow participants to interact in real-time by exchanging gaze behavior. To this end two MRI scanners were equipped with infrared eyetracking systems and video cameras. Thereby a live video image of the respective interaction partner's face could be broadcast into the respective other scanner to generate a mediated face-to-face situation (Fig. 5).

Figure 5.

Illustration of the combined hyperscanning and dual eyetracking setup used by Saito and colleagues (Taken from: Saito et al. Reference Saito, Tanabe, Izuma, Hayashi, Morito, Komeda, Uchiyama, Kosaka, Okazawa, Fujibayashi and Sadato2010).

In a simple task, participants were instructed to either look at (or away from) targets (blue or red balls) when a change in color indicated to do so or to follow (or not follow) the interactor's gaze toward a target. Using this setup for neuroimaging purposes, it was shown that “paired” subjects showed higher correlations than “non-paired” subjects in an area of the MNS, as assessed by performing an inter-brain correlation analysis of the blood-oxygen-level-dependent (BOLD) signal time courses as a measure of functional connectivity. Future investigations could use measures of effective connectivity to systematically explore both models of eye-movement control as well as measures of inter-brain synchronization (e.g., Anders et al. Reference Anders, Heinzle, Weiskopf, Ethofer and Haynes2011; Schippers et al. Reference Schippers, Roebroeck, Renken, Nanetti and Keysers2010). While we appreciate that the exploratory study by Saito et al. had to rely on a highly structured task, we see a significant number of possibilities for modifying and extending the paradigm and ingenious setup used by the authors.

Indeed, it is only very recently that researchers in social neuroscience have moved on to investigate (relatively) unconstrained real-time social interaction while measuring neural activity in both interactors (Fig. 1, C & D; Fig. 2: cube [5]). Dumas et al. (Reference Dumas, Nadel, Soussignan, Martinerie and Garnero2010) set out to investigate the inter-brain correlates of mutual adaptation processes in social interaction by recording with a dual-video and dual-EEG setup while interactors were asked to spontaneously imitate each other's hand movements. Results demonstrated that states of “interactional synchrony,” that is, states during which the hands of the two participants started and ended a movement simultaneously, correlate with a synchronized inter-brain network in the alpha-mu band in centro-parietal cortex. Interestingly, the alpha-mu band has been considered as an electrophysiological correlate of MNS functioning, and specific frequencies of this band have been discussed as a marker of social coordination (Tognoli et al. Reference Tognoli, Lagarde, DeGuzman and Kelso2007).

Against this background, we posit that a crucial achievement of future research in social neuroscience will consist in the investigation of the neural correlates of mutual adaptation and the reciprocity of human-human social interaction at the intra- as well as the inter-brain level (see Figs. 1 & 2). Beyond the specific methodological challenges regarding synchronized collection and integrated data analysis, a most fundamental challenge involved in this endeavor concerns the implementation of ecologically valid and at the same time highly controllable experimental settings, suitable both for behavioral and neural investigations (cf. Wilms et al. Reference Wilms, Schilbach, Pfeiffer, Bente, Fink and Vogeley2010; Zaki & Ochsner Reference Zaki and Ochsner2009). The state of the art in neuroimaging provides severe limitations to studying free-running interactions using the full range of verbal and nonverbal channels. Most likely successful studies will need to identify and isolate salient communicative subsystems. As referenced throughout this article, human gaze is certainly a most interesting candidate for this type of approach as it is a highly social cue, it can be unobtrusively measured and processed in real-time using modern eyetracking technology, and it can be easily experimentally controlled and dynamically displayed via artificial characters. Advantages of using virtual representations of human interlocutors in computer animations instead of real-life-interactions or video have long been discussed (see Bente Reference Bente1989; Blascovich et al. Reference Blascovich, Loomis, Beall, Swinth, Hoyt and Bailenson2002). Early studies using computer-animated characters in observational studies concerned with nonverbal behavior confirmed their external validity, for example, when comparing impressions of animated characters with video recordings of real people (Bente et al. Reference Bente, Krämer, Petersen and de Ruiter2001), and also demonstrated the enormous degrees of freedom in experimental control: for example, when overlaying the same nonverbal behavior on actors of different gender (Bente et al. Reference Bente, Feist and Elder1996) or neutralizing cultural stereotypes by using standardized animation “dummies” (Bente et al. Reference Bente, Leuschner, Al Issa and Blascovich2010). However, it is only recently that the advent of sophisticated capture devices for gaze, facial expressions, and body movement, as well as real time animation tools, allow for full character control in the framework of truly interactive research paradigms (Bailenson et al. Reference Bailenson, Blascovich, Beall and Loomis2003; Bente et al. Reference Bente, Rüggenberg, Krämer and Eschenburg2008b). Using these technologies in such an interactive paradigm, Bente et al. (Reference Bente, Eschenburg, Aelker and Moreno2007a; Reference Bente, Krämer, Eschenburg, Konijn, Utz, Tanis and Barnes2008a) used motion capture devices, data gloves, and eyetracking devices to capture the nonverbal behavior of two interlocutors and transmit this information in real time to the partner's screen where it was executed by an animated character. The whole setting thus established a “virtual video conference” (see Fig. 6).

Figure 6.

Basic version of an avatar-based interaction platform (screen shot during calibration; virtual characters shown on screen are mirroring the interlocutors own behavior).

Although movement was left unmodified, gaze was controlled by a computer program that systematically varied the duration of direct gaze. This kind of “blended interaction” allows for most realistic and lively displays of interaction while exerting strict experimental control over the specific cues under investigation. Furthermore, the computer program can be used to show specific responses conditional on the partner's behavior (e.g., to follow the partner's gaze or not, as in Schilbach et al. [2010b], to nod after sentence completion, and to show averted or directed gaze after specific key words or nonverbal signals). Beyond these unprecedented possibilities, animated characters are open for experimental manipulation and they also help to solve problems that arise from the application of complex technologies: Showing the interaction partner lying in a scanner or wearing an EEG cap, carrying reflectors for movement or face tracking, might decrease ecological validity more than meeting an avatar in a virtual environment. Measurement devices, however, do not only obscure our view of the interlocutor and provide an awkward image of the other; they also restrict the behavioral repertoire of the actors; for example, the locomotion radius when using a remote eye tracker or the mobility of extremities and head in the fMRI scanner, where actions are often limited to simple key presses. By providing options for an elaborate action repertoire within the constraints of neuroimaging settings, virtual reality or augmented reality setups are, therefore, likely to advance the investigation of the neural underpinnings of joint actions (Knoblich & Sebanz Reference Knoblich and Sebanz2008; Kokal et al. Reference Kokal, Gazzola and Keysers2009; Newman-Norlund et al. Reference Newman-Norlund, van Schie, van Zuijlen and Bekkering2007; Reference Newman-Norlund, Bosga, Meulenbroek and Bekkering2008). On the other hand, the use of (more) mobile data-collection devices, such as low-density EEG setups or methods such as functional near infra-red spectroscopy (fNIRS), could also help to overcome some of the bandwidth problems and create more ecologically valid situations, in which participants can use the full range of communicative behavior (e.g., Suda et al. Reference Suda, Takei, Aoyama, Narita, Sato, Fukuda and Mikuni2010; Reference Suda, Takei, Aoyama, Narita, Sakurai, Fukuda and Mikuni2011). Further, using mobile data collection and analysis devices could help to provide real-time visualizations of brain states and investigate how such interactionally presented neuromarkers might contribute to and impact on ongoing social interactions. Finally, another important matter is to investigate which insights can be gleaned from quantifying inter-brain effects of social interactions and to assess whether two interacting brains provide more information about the interaction than the sum of the individual brains (cf. Hasson et al. Reference Hasson, Ghazanfar, Galantucci, Garrod and Keysers2012; Konvalinka & Roepstorff Reference Konvalinka and Roepstorff2012).

Taken together, our review highlights the importance and potential of exploring the neurobiology of social cognition from an interactor's point of view and stresses that the use of more ecologically valid, truly interactive setups will help to address the neural bases of interpersonal coordination and responsiveness in interaction, which may underlie our seemingly effortless ability to apprehend other minds.

4.2.1. From implicit interaction to explicit mentalizing?

In establishing the relationship between social cognition from an interactor's as compared to an observer's point of view, the question of implicit and explicit processes in social cognition gains further importance. Till now it is largely unclear what the neural signatures of implicit as compared to explicit social cognition are (Frith & Frith Reference Frith and Frith2008). There is a theoretical as well as an empirical side to this issue, both of which can potentially be elucidated within a second-person account. In theoretical terms, it is not always clear what is meant by “implicit social cognition” and how this is actually related to awareness, on the one hand, and automaticity versus control, on the other. It is never spelled out whether “implicit social cognition” is just what we reason consciously about, minus awareness, or, if this is not the case, how exactly the former relates to the latter. For instance, social interaction may be conceptualized as a prime example of implicit learning, as is language, the development of both being closely intertwined (see sects. 2.2 and 2.3). As with implicit language learning, an important question is how we develop an explicit and symbolic, or rule-based grasp of language and social interaction based on implicit, predominantly associative learning of statistical regularities. If indeed such explicit knowledge is the result of exposure to increasingly complex social contexts, it remains unexplained how, and what the neural bases of such changes could be. For instance, how do behavioral precursors to TToM develop into a capacity to pass classic TToM tasks?

Ontogenetically, becoming an “expert” in social cognition may be, we suggest, primarily a question of social interaction, whereas later competencies of more detached, reflective social cognition might be a result of reactivating the networks forged during social interactions and re-describing these interaction-based capacities (cf. Cleeremans Reference Cleeremans2008; Cleeremans et al. Reference Cleeremans, Timmermans and Pasquali2007; Karmiloff-Smith Reference Karmiloff-Smith1992; Timmermans et al. Reference Timmermans, Schilbach, Pasquali and Cleeremans2012). In this respect, our proposal links up with theories about the functional structure of the brain, which take the “re-use” of neural circuitry for various cognitive purposes to be a central organizational principle (e.g., Anderson Reference Anderson2010). According to these theories, it is common for neural circuits to be established for one purpose and then to be put to different uses. Crucially, rather than seeing such a re-description as an internally generated, qualitatively different representation of discrete knowledge about the world, our second-person approach frames such a re-description as an ongoing learning process driven by increasingly complex interactive contexts, for example, when moving from dyadic to triadic interaction, which creates the possibility and need to somehow communicate with respect to an external, third object or person (see also Carpendale & Lewis Reference Carpendale and Lewis2004). Furthermore, the involvement of reward signals closely related to ongoing social interaction might also be relevant in this respect (Pasley et al. Reference Pasley, Mayes and Schultz2004). In the light of section 2.3, language in this context might not only be shaped by social interaction, but the other way around as well, with the gradual development of language providing a scaffolding that allows implicit social know-how to develop in explicit social knowledge. Indeed, as Hutto (Reference Hutto2008) suggests, the development from early nonverbal responding to others' mental states – which, according to Hutto, is essentially non-contentful – into explicit, contentful thinking about others' mental states is probably linguistically mediated. Future research in social neuroscience will have to find ways to systematically investigate how activity both in the parieto-frontal cortex and the so-called mentalizing network may be shaped and modulated by the degree to which human observers perceive themselves as participants of social encounters, and by actual exposure to social interactions; and to understand how this relates to recruiting these networks during observation (Schilbach Reference Schilbach2010, cf. Wang et al. Reference Wang, Ramsey and Hamilton2011).

Apart from investigations of real-time dyadic and triadic interaction in healthy persons, we also see great potential for investigating real-time social interaction during development and in diagnostic groups. In spite of the importance of joint attention in ontogeny and a growing number of neuroimaging studies in this respect, the neural correlates of this significant phenomenon and how it unfolds in real-time are incompletely understood. Given that our paradigm does not rely upon verbal information and higher-order reasoning about others' mental states, but relies upon naturally occurring social behavior, it might prove to be particularly useful in this respect and could help to address how human infants come to enjoy the sharing of experiences. Furthermore, a characterization of the neural signature of mutual and joint attention in infancy could be related to the development and neural bases of other social cognitive abilities that develop later in life, to investigate commonalities and differences in a longitudinal setup. Such an approach could help to directly test whether interaction-based network modulations are later co-opted and reused for higher-order, explicit processes.

A litmus test for the second-person approach in ontogeny would be to investigate infants' responses to acts of mentality (e.g., intentional actions, attention, emotional facial expressions), depending upon the infant's ability to respond to them when they receive them. Contrary to a simulationist reading, which would predict that infants need to be able to do the actions themselves, and to “mind-reading” accounts, which hold that the infant needs to be able to observe and conceptualize patterns in order to be able to grasp actions meaningfully, the second-person approach suggests that acts of mentality should be meaningful to infants depending on the infant's ability to respond to them when they receive them, regardless of whether the infants can do the actions themselves.

Thus, a second-person neuroscience is concerned with the question of how direct social interaction with others – and the implicit processes inherent to this – may inform and modulate more detached, explicit perspectives on them. Evidence from developmental psychology has been taken to suggest that reflective (or “meta-cognitive”) competence emerges through the infant's social interactions with others by relating one's own states with those of others (e.g., Rochat & Striano Reference Rochat and Striano2002). However, a more radical approach holds that it may be only via social interaction and in virtue of the fact that we are constantly trying to model other minds in interaction that we learn to be conscious and develop both an understanding of ourselves and a conscious percept of the world at all (Cleeremans Reference Cleeremans2011; Timmermans et al. Reference Timmermans, Schilbach, Pasquali and Cleeremans2012). A study by Bahrami et al. (Reference Bahrami, Olsen, Latham, Roepstorff, Rees and Frith2010) further indicates that such interaction-based improvements of introspective abilities continue to be effective during adulthood.

4.2.2. From mentalizing and mirroring to automatic interaction?

A question equally important to an ontogenetic account is how we manage to engage in interaction on a daily basis, without referring to explicit knowledge. Instead, we interact and “go with the flow,” all the while retaining the capacity to reflect upon our interlocutor and ourselves afterward. It is exactly here that neurobiological evidence of TToM-related activations has given rise to the idea of “implicit TToM” or “implicit social cognition.” But if this “implicit TToM” is essentially something different than, say, what we consciously deploy in False Belief tasks, then what does it represent? It is here that the most immediate empirical questions may be raised: Are the implicit and explicit processes that occur in interaction actually the same processes as those that occur during observation? We suggest that this may not be so, in that interaction-based processes involve at least two aspects. First, they involve an automatic inference component based on previously acquired knowledge, which is also present during observation, and which is largely impervious to explicit modification. Second, they also require an action-control component, which can be fast (<350msec) and based on the previously acquired knowledge, or slightly slower (>350msec) and potentially available to control (Khan et al. Reference Khan, Lawrence, Fourkas, Franks, Elliott and Pembroke2003; Pisella et al. Reference Pisella, Gréa, Tilikete, Vighetto, Desmurget, Rode, Boisson and Rossetti2000; Rossetti et al. Reference Rossetti, Pisella and Vighetto2003; Sartori et al. Reference Sartori, Becchio, Bulgheroni and Castiello2009). As such, whereas during interaction implicitly activated knowledge seems to be immune to explicit interference (Ambady et al. Reference Ambady, Bernieri and Richeson2000; Choi et al. Reference Choi, Gray, Ambady, Hassin, Uleman and Bargh2005), automatic processes are – due to their action component – susceptible to explicit modification, or interference (you drive a car or play a musical instrument without thinking about particular movements contributing to it, but you can to some degree bring them under conscious control – which under normal circumstances will hamper fluent performance). However, such awareness of action is related not only to action preparation (motor signal), but is additionally prone to strong reconstructive effects of the sensory consequences of one's actions (Haggard Reference Haggard2009; Haggard & Tsakiris Reference Haggard and Tsakiris2009; Pacherie Reference Pacherie2008; Sarrazin et al. Reference Sarrazin, Cleeremans and Haggard2008; Sato & Yasuda Reference Sato and Yasuda2005). In an observational context such factors simply do not come into play, whereas in an interactive context, this means that the reactions of the other co-determine the degree to which one becomes conscious of one's actions, to which anyone who ever committed a serious gaffe in public will readily testify.

An illustration of how the reasoning described above might lead to novel approaches lies in psychiatric disorders: For instance, autistic persons have been described as having problems with implicit or intuitive social cognition. However, in terms of elementary processes, several studies have been unable to find problems with implicit learning (Brown et al. Reference Brown, Aczel, Jimenez, Kaufman and Plaisted-Grant2010), action representation (Sebanz et al. Reference Sebanz, Knoblich, Stumpf and Prinz2005), or even mirroring (Southgate & Hamilton Reference Southgate and Hamilton2008). Thus, one empirically verifiable possibility would be that, at least in HFA, both low-level and high-level processes are relatively intact, but that the application of this implicit know-how versus explicit knowledge is disturbed, in that persons with HFA apply explicit, high-level TToM in situations where healthy controls “trust a gut feeling.” This then could lead to the hypothesis that HFA persons lack the know-how or knowledge that would subtend such a gut feeling (their explicit knowledge not being rooted in interactive know-how), or, alternatively, that they fail to recognize and trust such a gut feeling and instead, in any task allowing explicit reasoning, will opt for such explicit rules instead (failure of automatic application of knowledge, whether know-how or explicit knowledge). Crucially, the notion of how these potentially different capacities, implicit learning and automatic behavior contribute to autism – and healthy controls – remains elusive in passive spectatorial settings.

4.2.3. Motivation and the spark to interaction: Putting reward signals into social cognition

Another important feat will be to address how motivation- and reward-related signals could be differentially engaged during participation in social situations as compared to observation of others' interaction and might interact with brain regions relevant for action control (cf. Oullier & Basso Reference Oullier and Basso2010; Pessoa & Engelmann Reference Pessoa and Engelmann2010; Schonberg et al. Reference Schonberg, Fox and Poldrack2011; Fig. 1). This directly follows from our conception of “active” social perception strongly relying upon affective and reward-based contingencies that result from being personally addressed and from prompt responses which feed into ongoing social interaction. Being in the presence of someone, human beings cannot help but interact with each other (e.g., Zwickel & Vo Reference Zwickel and Vo2010), which may be due to an evolutionary heritage of depending upon others for survival; possibly having rendered social interaction “intrinsically” rewarding (Coan Reference Coan, Cassidy and Shaver2008; Tomasello Reference Tomasello2009). Consequently, being a participant (as compared to being an observer) of social interaction, we speculate, might also lead to an expectation of reciprocation that is more in line with cooperative than competitive behavior and motives (Pfeiffer et al. Reference Pfeiffer, Timmermans, Bente, Vogeley and Schilbach2011). Furthermore, human beings appear to be extremely sensitive to having an impact on conspecifics and seem to enjoy exerting such influence from early on in life (Fiske & Dépret Reference Fiske and Depret1996; Gergely & Watson Reference Gergely and Watson1996; Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b). Relatedly, human beings not only enjoy being imitated by others (as long as this is done covertly), but this also increases feelings of affiliation and rapport (Chartrand & Bargh Reference Chartrand and Bargh1999). Consistently, experiences or even hints of social exclusion (even third-party exclusion) increases affiliative imitation (Over & Carpenter Reference Over and Carpenter2009). One of the factors that makes exerting an influence on other persons so rewarding, might be their agentive nature: First, it vastly increases the action possibilities of the other, and hence, the uncertainty of what the effect of my action will be (compared to when I interact with, say, a vending-machine); second, it might be that simply knowing that the other is an independent agent having other options than reacting to you, makes a contingent response very rewarding.

To investigate interactions of motivation- and reward-related signals with activity in action control and social cognition networks during participating in social interaction, we have devised a non-verbal Turing test, which is based on the interactive eyetracking paradigm used to investigate joint attention described in sect. 3.1.2. In this setup participants engage in gaze-based interaction with an anthropomorphic virtual character, whose gaze behavior is said to be controlled by another participant or by a computer, while, in fact, it is always controlled by a computer algorithm. Using this setup, the character's gaze reactions were systematically varied along a continuum from a maximal probability of gaze aversion to a maximal probability of gaze-following during brief interactions, thereby varying congruency and contingency of the reactions. We investigated how these variations influenced whether participants believed the character to be controlled by a person or a computer. In a series of experiments, the confederate was either introduced as “naïve” to the task, “cooperative” or “competitive.” Results demonstrate that the ascription of agency increases with higher congruency of gaze reactions when participants are interacting with a “naïve” partner. In contrast, agency ascription was driven by the degree of contingency (i.e., the probability of the virtual other reacting) irrespective of congruency (whether or not the behavior shown in response was the same or different than the participant's) reactions when the confederate was introduced as “cooperative.” Conversely, during interaction with a “competitive” confederate, judgments were neither based on congruency, nor on contingency (Pfeiffer et al. Reference Pfeiffer, Timmermans, Bente, Vogeley and Schilbach2011). These results offer important insights into what renders the experience of an interaction truly social: Humans appear to have a default expectation of reciprocation as evidenced by the naïve condition, which can be influenced drastically by the presumed disposition of an interactor to cooperate or compete. This finding, we assume, could be related to our neuroimaging results, which were accrued by making use of the same kind of paradigm, and highlight the involvement of reward-related neurocircuitry during congruent interactions (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b). The use of the Turing test paradigm for neuroimaging purposes is likely to advance our understanding of the neural bases of social interaction: For example, it can be hypothesize dthat gaze-based interactions with a naïve confederate might lead to an increase in neural activity in areas of the so-called mentalizing network, whereas conditions with highly congruent reactions might lead to an increase in reward-related brain areas. Although competitive interactions might also lead to an increase of neural activity in mentalizing areas, it will be interesting to investigate whether a competitive context could lead to a decrease of activity in reward-related neurocircuitry. Moreover, the question can be targeted whether the presence of a shared goal (as in the cooperative context), leads to a decrease of activity in the mentalizing system, while concomitantly an increase of activity might be observed in regions that have been implicated in coordinated behavior. Furthermore, it will be interesting to investigate whether changes in activity of the brain's reward system in response to positively contingent gaze-reactions could generalize to contingent reactions irrespective of their valence depending on the situational context. Finally, neuroimaging studies, which make use of this paradigm, may help to investigate putative connectivity differences between relevant brain areas during social interaction (Fig. 1). This could help us to understand the neural bases of opposing effects between imitative behavior and understanding of others' minds (Stel et al. Reference Stel, van Dijk and Olivier2009).

Recent evidence suggests that different reward types recruit the reward system of the brain, which has given rise to the suggestion of a ‘“common neural currency’” for rewards (Izuma et al. Reference Izuma, Saito and Sadato2008). The neural mechanisms of differences in the susceptibility for different types of reward, however, remain unclear. Also, it is not well known how different reward types may compete and can act as a surrogate for one another (e.g., using food rewards when other rewards are unavailable) or how social influences may serve as a control mechanism to reward dependency. In order to probe the reward system of the brain while at the same time allowing for the use of different reward stimuli, the interactive eyetracking paradigm could be adapted to serve as a reward-based saccade paradigm. In such a paradigm, participants would be asked to generate and be rewarded for successfully completing saccades to visual targets. Rewards could be varied to include monetary, food, and social rewards (congruent gaze reactions of an anthropomorphic virtual character also shown on the stimulus screen). On the behavioral level, higher reward probability for a given target is expected to lead to a decrease in saccade latencies (e.g., Milstein & Dorris Reference Milstein and Dorris2007), the magnitude of which may depend upon the specific reward type. On the neural level, the correlates of this reward-related bias and possible reward type-specific differences in controls and patient groups characterized by social impairments (such as patients with autism and schizophrenia) are not well known. Research in nonhuman primates suggests that midbrain dopamine neurons involved in motivated saccade tasks receive inputs from many brain areas including the ventral and dorsal striatum, the subthalamic nucleus, the amygdala, the lateral hypothalamus, the dorsal raphe nucleus, and the lateral habenula (Bromberg-Martin et al. Reference Bromberg-Martin, Hikosaka and Nakamura2010; Matsumoto & Hikosaka 2009). A comparison of different reward types could not only allow us to investigate overlap and differences in the underlying neural correlates, but could also enable the calculation of a scaling factor (i.e., an “exchange rate”) between social, monetary, and food rewards for each subject.

4.3.1. Computational social neuroscience

Although computational neuroimaging studies inspired by game theory and other approaches that use generative models for data analyses have already taken up the investigation of competitive scenarios and are beginning to address cooperative games (e.g., Yoshida et al. Reference Yoshida, Dolan and Friston2008), human beings' unparalleled abilities for cooperation in real-time social interaction have largely been left unexplored. Here, extending computational social neuroscience approaches to reciprocal and adaptive two agent-interactions holds great potential for future research and may enable us to investigate how differences in perspective towards others, that is, social cognition from an agent's versus from an observer's point of view, may influence the reliance on different computational mechanisms. For example a study by Hampton et al. (Reference Hampton, Bossaerts and O'Doherty2008) demonstrates the importance of representing the possible influence one may have on others in direct interaction. Similarly, recent findings from model-based neuroimaging studies already do provide evidence for differences in the neural mechanisms of experienced as compared to observational learning (activity changes in ventral striatum being selective to instrumental actions; Cooper et al. Reference Cooper, Dunne, Furey and O'Doherty2012). Computational approaches are particularly powerful in helping to address the history of an ongoing social interaction, how learning may occur and how it could be related to a social reward prediction error. In doing so, model-based analysis of neuroimaging data could help to investigate the development of neural networks and their structural and neurofunctional underpinnings during social interaction, thereby helping us understand how particular functions develop in particular areas of the brain or, specifically, the brains of interacting minds. One example of how a computational social neuroscience paradigm may help to assess and quantify the impact of (gaze-based) social interactions is the following: By combining a cooperative game such as the “stag hunt” game developed by Yoshida et al. (Reference Yoshida, Dolan and Friston2008) with a gaze-based social interaction manipulation (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b), it becomes feasible to test computational models under more ecologically valid conditions. In light of our hypothesis and preliminary findings, which suggest that social interactions may prompt collaborative motives, we expect that congruent gaze behavior exhibited by the virtual character in our joint attention manipulation may promote cooperative behavior in the “stag hunt” game, whereas gaze aversion of the virtual character may have the opposite effect. Using fMRI while participants are involved in gaze-based interactions and in the stag hunt game may also allow to use correlational approaches and multi-variate pattern analysis to predict the level of cooperation in the stag hunt from neural activity observed during gaze-based interaction.

Alternatively, future studies could use more elaborate models to control the behavior of a virtual character (conversational agents) in more complex ways (e.g., Broz et al. Reference Broz, Nourbakhsh and Simmons2013; Kelso et al. Reference Kelso, de Guzman, Reveley and Tognoli2009; Triesch et al. Reference Triesch, Teuscher, Deak and Carlson2006; Reference Triesch, Jasso and Deak2007; Fairhurst et al. Reference Fairhurst, Janata and Keller2012). The meaningful use of conversational agents (i.e., completely computer generated artificial social entities) in interaction studies, in contrast to avatars, would require the implementation of basic social skills in the agents, serving the perception, processing and production of interactive behavior (see Vogeley & Bente Reference Vogeley and Bente2010). Such implementations are feasible if the scope of behavior is constrained with regard to channel bandwidth (e.g., to eye gaze), as well as with regard to the task affordances (e.g., fixation of objects, gaze following, etc.). However, the available technologies are far from allowing for broadband interactions in open conversational situations. Using such an approach might provide a most informative complementarity to recent investigations, which have focused on using computational models to predict neural responses in a human observer or interactor (e.g., Behrens et al. Reference Behrens, Hunt and Rushworth2009; Cooper et al. Reference Cooper, Dunne, Furey and O'Doherty2012) and could help in exploring the explanatory power of mutually adaptive modeling approaches.

4.3.2. Genotype-specific effects and hormonal influences of the neural basis of social interaction

Recent evidence suggests that important hormonal influences exist which can specifically affect social behavior and its underlying neuroanatomical and neurofunctional correlates (e.g., Chura et al. Reference Chura, Lombardo, Ashwin, Auyeung, Chakrabarti, Bullmore and Baron-Cohen2010; Hermans et al. Reference Hermans, Bos, Ossewaarde, Ramsey, Fernandez and van Honk2010; Volman et al. Reference Volman, Toni, Verhagen and Roelofs2011), giving rise to the notion of a field of “social neuroendocrinology” (cf. Bos et al. Reference Bos, Panksepp, Bluthe and Honk2012). Interestingly, it has been suggested that certain hormonal responses only occur in ecologically valid situations (e.g., Henckens et al. Reference Henckens, Hermans, Pu, Joels and Fernandez2009), which is consistent with the assumptions of the here proposed second-person approach. Therefore, it seems likely that using the types of interactive and ecologically valid paradigms endorsed by the second-person account may also help to advance the investigation of the complex interplay and influence of hormones on the “social brain.” In addition, efforts are also being undertaken to understand genotype-specific differences in prosocial hormones and brain anatomy and how they impact on social behavior (e.g., Chen & Johnson Reference Chen and Johnson2012; Tost et al. Reference Tost, Braus, Hakimi, Ruf, Vollmert, Hohn and Meyer-Lindenberg2010). Furthermore, we see great potential in elucidating how such genotype-specific differences might be related to differences in reward-based processing (e.g., Chakrabarti & Baron-Cohen Reference Chakrabarti and Baron-Cohen2011) and could predict differential responses to pharmacological challenges or therapeutic interventions. In the following, we therefore go on to discuss the possible implications of a second-person approach to other minds for our understanding of psychiatric disorders.

5.2.1. The impact of social gaze on action control and interpersonal coordination in high-functioning autism

Autism is characterized by qualitative impairments of social interaction and communication. Here, the distinction between implicit and explicit processes has recently been emphasized: Contrary to popular explanations, what seems to be impaired in high-functioning autism (HFA) is not the ability to use explicit mentalistic inference, but rather, the implicit processes that contribute to participating in social interaction and that allow us to orient towards, and automatically integrate, relevant social cues in more complex situations (Becchio et al. Reference Becchio, Pierno, Mari, Lusher and Castiello2007; Kuzmanovic et al. Reference Kuzmanovic, Schilbach, Lehnhardt, Bente and Vogeley2011; Senju et al. Reference Senju, Southgate, White and Frith2009). Individuals with HFA are neither impaired in explicit social cognition – as they, in fact, consciously remind themselves to think about the mental states of others in an attempt to compensate for interaction difficulties – nor in their capacity of implicit learning in general (Brown et al. Reference Brown, Aczel, Jimenez, Kaufman and Plaisted-Grant2010). Instead, they may lack the more basic skills to respond intuitively to socially relevant information by generating “adaptive actions upon salient aspects of the environment” (Klin et al. Reference Klin, Jones, Schultz and Volkmar2003). The discrepancy between the ability to solve explicit social cognitive problems and the inability to meet the demands of everyday life social interactions in HFA has been highlighted by methods which enable probing involuntary aspects of behavior. The difference between explicit and implicit social cognition is also nicely captured by descriptions of subjective experiences that persons with HFA provide. Here, individuals with HFA emphasize a preference for being in an observational stance towards others as this allows them to “logically compute,” “contemplate,” and “think through” aspects of the situation in a conscious and cognitively effortful way. Furthermore, being an observer also allows one to think about what one may know about a given situation and the constraints this may entail for a person who is actually in it. On the contrary, the situation becomes remarkably different and more complicated for a person with HFA when the person she has quietly been observing turns towards her and tries to engage her in interaction. Here, patients describe “being flooded” by information, an “inability to keep up,” and “not knowing when and how” to respond to what others do possibly due to the imperfect contingencies inherent in social interaction (Klin et al. Reference Klin, Lin, Gorrindo, Ramsay and Jones2009).

We suggest that the impairments of being able to respond intuitively to socially relevant information could be closely related to the ability to perceive social affordances, that is, the possibilities for interaction provided by others. In order to investigate the impact of social cues on mechanisms of action control, we therefore made use of a stimulus-response compatibility (SRC) paradigm, in which control and HFA participants were asked to generate spatially congruent or incongruent motor responses to changes in a face, and to a face-like and an object stimulus (Schilbach et al. Reference Schilbach, Eickhoff, Cieslik, Kuzmanovic and Vogeley2012b). Results demonstrate that whereas in control participants being looked at by a virtual other leads to a reduction of reaction-time costs associated with generating a spatially incongruent response, this effect is not present in the HFA group. We suggest that this modulatory effect of social gaze on action control in control participants might be related to social cues triggering motor preparatory programs that may help to coordinate one's actions with those of someone else, rather than providing a mere “resonance” mechanism. It is conceivable that participants thereby automatically adjust to the “perspective” of the virtual agent by making sense of the situation in pragmatic terms (Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006). Hereby, an action that is deemed spatially “incongruent” from an allocentric observer's point of view might end up being “complementary” to an action that the participant perceives the (virtual) other perform. In other words, being in the presence of someone else (even a virtual other) may change our perception of the environment towards perceiving it in terms of the resources or possibilities for actions held collectively (“we-space”), rather than individually (e.g., Krueger Reference Krueger2010).

Indeed, consistent with this interpretation of social cues leading to a recruitment of motor-preparatory and action-monitoring-related processes, an fMRI study from our lab using an abridged version of the social SRC paradigm in a group of healthy participants demonstrated that performing incongruent actions in the presence of a virtual other (as compared to performing them in response to a nonsocial stimulus) leads to a differential increase of neural activity in the brain regions known to be involved in action monitoring and response inhibition (anterior cingulate cortex and inferior frontal gyrus), and also in areas previously implicated in social cognition (dorso-medial prefrontal cortex; see Schilbach et al. [2010a] for details). Furthermore, reaction time differences between the social and nonsocial condition appear to be closely related to activity changes in the reward system of the brain as measured by BOLD fMRI: Participants who demonstrated more pronounced activations of the ventral striatum – often related to hedonic or rewarding experiences – also found it more difficult to disengage from the face stimulus in order to generate incongruent motor responses, which is consistent with suggestions of an important influence of reward-related processes on mechanisms of action control (cf. Pessoa & Engelmann Reference Pessoa and Engelmann2010). Taken together, these results highlight that performing actions in a – albeit minimal – social context significantly affects mechanisms of action control (Schilbach et al. Reference Schilbach, Eickhoff, Cieslik, Shah, Fink and Vogeley2010a; Schilbach et al. Reference Schilbach, Eickhoff, Cieslik, Kuzmanovic and Vogeley2012b; Gangopadhyay & Schilbach Reference Gangopadhyay and Schilbach2012).

Our findings are consistent, therefore, with the proposed difficulties in the automatic integration of social information in HFA, whereas more explicit measures of social cognition may be intact as a result of compensatory strategies (Senju et al. Reference Senju, Southgate, White and Frith2009; Kuzmanovic et al. Reference Kuzmanovic, Schilbach, Lehnhardt, Bente and Vogeley2011). Whereas control participants respond to the virtual agent's gaze by recruiting motor-based processes, which lead to a performance benefit as and by recruiting the brain's reward system, which is likely relevant for the hedonic experience of social stimuli, our results seem to suggest no such differences in HFA. In other words, individuals with autism may not lack a theory of how other minds work (“knowing that”), but the relevant know-how for dealing with others in direct interaction (cf. Zahavi & Parnas Reference Zahavi and Parnas2003). Also, important differences are likely to exist in the degree to which social interactions can evoke comparable responses in reward-related neurocircuitry in patients with HFA (cf. Schultz Reference Schultz2005).

In line with the second-person approach, future research should, therefore, investigate impairments of social cognition in autism as reflecting difficulties in skillfully interacting with others, as those might be at the very heart of the problems faced by patients in everyday social encounters. We suggest that the investigation of the neural mechanisms underlying these clinically relevant differences in high-functioning autism will benefit substantially from using more ecologically valid paradigms that look at how people actively engage and interact with one another in social encounters. Furthermore, it will be important to relate known differences in structural (Catani et al. Reference Catani, Jones, Daly, Embiricos, Deeley, Pugliese, Curran, Robertson and Murphy2008) and functional connectivity in autism (Kennedy et al. Reference Kennedy, Redcay and Courchesne2006) to interaction abilities. Here also, connectivity differences in the mentalizing or default mode network (see sect. 3) and the anti-correlated fronto-parietal network, as well as interactions of both networks with the reward system of the brain (Camara et al. Reference Camara, Rodriguez-Fornells and Munte2008), appear as primary candidates for such studies. Using paradigms that enable control of the bandwidth of the social interaction may help to titrate levels of perceptual complexity and differences in the temporal structure of an interaction, in order to adapt to the differences in functional and structural connectivity and concomitant interaction requirements. Also, using mediated social interactions could help to provide additional cues for patients, which help them to engage in interaction.

5.2.2. Language use in high-functioning autism

According to the view of “language-as-action” or of language as a “tool for social interaction” (Tylén et al. Reference Tylén, Weed, Wallentin, Roepstorff and Frith2010), language processing can be regarded as a joint activity which results in entrainment and synchronization between partners on various levels (Pickering & Garrod Reference Pickering and Garrod2004). Furthermore, this “grounding” view of language suggests that words do not contain their meaning, but that linguistic labels can be highly negotiable and that interlocutors flexibly seek to understand them against the background of a “common ground” (Clark Reference Clark1996). To this end interlocutors produce and monitor paralinguistic cues and one another's instrumental behavior to ensure that they, indeed, understand each other. This view of language processing in dialogue – whereby a partner's contribution plays a dual role by providing the other person with evidence of how the previous utterance has been understood – resonates with our proposal of a second-person approach to other minds, which also highlights the importance of reciprocal relations and their partially constitutive role in interpersonal understanding. Here, the various feedback from interaction partners and bouts of interpersonal synchronization shape and structure the “interaction space” and the denouement of the social encounter.

In high-functioning autism several language-related abnormalities are present, which, we believe, can be best understood in terms of the “grounding” rather than the “message model” view of language processing: Individuals with high-functioning autism – in spite of their elaborate linguistic code, and the absence of developmental language delay in Asperger's syndrome – have profound difficulties in understanding irony and insist on the use of the literal meaning of words, for which a more “objective,” rule-based, situation-independent and “correct” usage can be determined. Consequently, the language in high-functioning autism is characterized by pragmatic and semantic deficits with patients being less likely to integrate contextual information (Groen et al. Reference Groen, Tesink, Petersson, van Berkum, van der Gaag, Hagoort and Buitelaar2010). Consistently, patients seem to have particular difficulties in adapting to the changes in linguistic labels in reference to an immediately given social context (Nadig et al. Reference Nadig, Vivanti and Ozonoff2009). Whereas in normal controls language promotes interpersonal coordination by helping to coordinate attention even in the physical absence of an interlocutor (Richardson et al. Reference Richardson, Dale and Kirkham2007a; Tylén et al. Reference Tylén, Weed, Wallentin, Roepstorff and Frith2010), the same does not appear to be true for individuals with high-functioning autism. Patients commonly describe difficulties in maintaining telephone conversations on account of the fact that they find it nearly impossible to know when to speak in the absence of visual cues. In summary, while the predominantly information-based functions of language (as described by the “message model” of language) seem to be intact in high-functioning autism, it is the socially constructed and interactionally coordinative functions of language, which operate on a more implicit and automatic level, that appear to be most impaired (Pexman et al. Reference Pexman, Rostad, McMorris, Climie, Stowkowy and Glenwright2011).

5.2.3. Meeting the interaction requirements of patients with high-functioning autism

From a clinical perspective, individual reports of patients with HFA from our outpatient clinic are also noteworthy, in that patients describe not having any significant impairments of social interaction and communication when they interact with other persons diagnosed with HFA, which stands in striking opposition to the difficulties that systematically occur when they interact with persons without this diagnosis. While also raising issues about the normative aspects of psychiatric diagnoses, these anecdotal reports emphasize that successful interaction and communication are crucially a matter of interpersonal adjustments and (possibly) an awareness of the interactional requirements of another person. Investigating these issues in patients with autism, we suggest, might help to improve not only psychotherapeutic interventions, but could also help to substantially raise public awareness, thereby promoting flexibility and openness to diversity in styles of social interaction. This is especially crucial for the case of HFA: As we are normally not aware of our intuitive, nonverbal capacities that are automatic in nature, we do not even have the chance to take notice of the specific deficits in the communicative behavior of HFA patients. With respect to the focus of this article, namely, the scientific investigation of real-time social interactions, it is most interesting to contrast dyads of neurotypical persons with neurotypical-patient and patient-patient dyads. Investigations of the latter type might help to shed light on the neural mechanisms that enable successful interactions between persons known for their social interaction impairments. Also, the study of differences in social interaction dynamics could prove to be a promising route and might be developed into a tool for helping differential diagnosis in psychiatric disorders.