Within their framework, Clark and Fischer (C&F) focus on observable (e.g., language) and self-reported behaviour (e.g., ratings on a questionnaire). While these measures provide a first indication of how people perceive and interact with social robots, they do not paint a complete picture. We propose that perspectives and techniques from psychology and neuroscience will not only allow to answer if people indeed represent a social robot at three connected physical scenes but also when and how. More objective measures, such as neuroimaging, can peal apart the multiple layers of human–robot interactions by outlining the neural and behavioural mechanisms supporting these interactions.

When observing the emotions expressed by a social robot, an individual could be asked what emotion the robot is displaying (i.e., open-ended question) or if the robot is happy or angry (i.e., two-alternative forced choice). While these and similar subjective measures (e.g., questionnaires, in-depth interviews) already provide a glimpse of how the individual views the robot, they provide just that, a glimpse. The indication of an emotion does not mean that the robot is represented as a happy robot. Nor does it mean that the same mechanisms are used to observe and understand the emotions expressed by the robot as when people observe and understand the emotions of other people. Behavioural and neural measures are vital to truly understand the social cognitive mechanisms during human–robot interaction (Wiese, Metta, & Wykowska, Reference Wiese, Metta and Wykowska2017). Advanced neuroimaging and brain stimulation techniques paired with new analytic approaches as well as implicit measures will provide a more detailed understanding of the representation held by the human user. For instance, fMRI studies indicate that some neurocognitive processes, such as person perception, show similar profiles across interactions with people and social robots, while other processes, such as theory-of-mind, show dissimilar profiles during these interactions (Hortensius & Cross, Reference Hortensius and Cross2018; Hortensius et al., Reference Hortensius, Hekele and Cross2018; Wykowska, Chaminade, & Cheng, Reference Wykowska, Chaminade and Cheng2016). It is important to note that even in the presence of similar behavioural patterns or neural activity, the underlying mechanism might differ between these interactions. New analytic approaches derived from cognitive neuroscience, such as representational similarity analysis, can test if behavioural reactions or activity within a neural network represents, for example, agent type (robot or human) or emotion (angry or happy) (Henschel, Hortensius, & Cross, Reference Henschel, Hortensius and Cross2020). These techniques and approaches could therefore be vital in outlining at what level and scene people represent the robot expressing emotions.

This approach has been successful in distilling the multiple layers during human–robot interactions, for example during mind perception. Top-down effects of mental state attribution are widely observed in social perception (Teufel, Fletcher, & Davis, Reference Teufel, Fletcher and Davis2010) and this also holds for human–robot interaction. Besides the appearance of the robot, the beliefs and expectations held by the individual play a critical role how they construe the social robot (Hortensius & Cross, Reference Hortensius and Cross2018; Wykowska, Wiese, Prosser, & Müller, Reference Wykowska, Wiese, Prosser and Müller2014). For example, if people believe that the action of the robot has a human-origin, activity in a region of the theory-of-mind network is increased compared to when people believe the action has a pre-programmed origin (Özdem et al., Reference Özdem, Wiese, Wykowska, Müller, Brass and Overwalle2017). When teasing apart mind perception even further, we can dissociate two distinct processes: theory-of-mind and anthropomorphism. Often viewed or treated as similar or even as one form of mind perception, recent evidence from psychology and neuroscience suggests otherwise (Hortensius et al., Reference Hortensius, Kent, Darda, Jastrzab, Koldewyn, Ramsey and Cross2021; Tahiroglu & Taylor, Reference Tahiroglu and Taylor2019). While the observed outcome, understanding the actions and hidden states of an agent, is the same, these forms of mind perception are likely supported by separate behavioural and neural mechanisms. For instance, activity within the theory-of-mind network did not relate to an individual's tendency to ascribe human characteristics to objects and nonhuman agents, such as robots (Hortensius et al., Reference Hortensius, Kent, Darda, Jastrzab, Koldewyn, Ramsey and Cross2021). Even if the observer understands the gestures and motion of the robot as happy, it does not mean that they truly belief that the robot is happy or that the same processes are used as when understanding the happiness of a friend.

Not only can this approach elucidate the neural and behavioural mechanism supporting human–robot interactions, it can also indicate the reliance of these interactions on both social and non-social processes. Besides the possibility that the human user represents the robot as a depiction of a social agent, it is also possible that it is represented as a depiction of an object. The main focus of human–robot interaction research has mostly been on if robots activate similar neurocognitive processes as humans. The reference or comparison category in this case is thus always a human agent, thereby restricting the focus on neurocognitive processes that are social in nature. Considering to what extent human–robot interactions rely on non-social neurocognitive processes or processes that extent over domains (e.g., attention, memory) is vital (Cross & Ramsey, Reference Cross and Ramsey2021). Robust activation in object-specific brain regions has been observed across neuroimaging studies on the perception and interaction with robots (Henschel et al., Reference Henschel, Hortensius and Cross2020). Extending the scope of both neurocognitive mechanisms and reference and comparison categories is needed, to understand if people construct these agents as (depictions) of objects or social agents (including animals), or as a completely new, standalone category. It is unlikely that one category fits all, as for instance, not only appearance and behaviour of the robot influence social cognitive processes (Abubshait & Wiese, Reference Abubshait and Wiese2017; Waytz, Gray, Epley, & Wegner, Reference Waytz, Gray, Epley and Wegner2010), but also context (e.g., lifelikeness of the interaction) (Abubshait, Weis, & Wiese, Reference Abubshait, Weis and Wiese2021). Importantly, people can hold different views of a robot. For example, implicit and explicit measures of mind perception do not correlate (Li, Terfurth, Woller, & Wiese, Reference Li, Terfurth, Woller and Wiese2022). It is therefore possible that people can view a robot as an object while in parallel view the robot as a social entity ostensibly experiencing happiness.

Together, this psychology and social and cognitive neuroscience approach to the study of human–robot interaction will provide a much completer picture by providing the necessary evidence for or against the framework put forward by C&F, and ultimately tell if, when, and how people construe social robots as mere depictions of social agents.