Niedenthal et al. define embodied simulation as “a simulation of a state in the motor, somatosensory, affective, and reward systems that represents the meaning of the expression to the perceiver” (sect. 1, para. 4, italics in the original). In this sense, the state of the sensory-motor system classifies the smile (“represents the meaning”), but the meaning of the classification to the perceiver must also be grounded. Although prototypical facial expressions may be grounded in intrinsic reward systems, the subtleties of non-prototypical smiles likely must be learned and, therefore, may be grounded in the socio-behavioral context surrounding the smile: Smiles are not presented or perceived in isolation; they are surrounded by rich socio-behavioral context. For example, it is difficult to consider the meaning of dominance smiles without reference to more than facial expressions.
Like other forms of communication, smiles can have profound indirect consequences. For example, Sharlemann et al. (Reference Scharlemann, Eckel, Kacelnik and Wilson2001) investigated how smiles can be used to negotiate cooperation between humans. Smiles (and, more generally, facial expressions) provide clues about the values and objectives of other agents. Because smiles have only negligible direct physical impact, the meaning of a smile cannot be fully realized by considering only facial sensory-motor systems. Instead, we propose that an important component of the meaning of a smile must be grounded in high-level behaviors. But this raises a question: How can the meanings of smiles be grounded in high-level behaviors?
Choe and Smith (Reference Choe, Smith, Gil and Mooney2006) present a framework that explains the process of learning the meanings of internal sensory states by grounding them in terms of specific patterns of motor action, actions that maintain invariance in the internal sensory state. Our Figure 1 shows the framework, and Figure 2 shows how invariance in internal state can link up with motor-based meaning. The action that maintains invariance in the agent's internal sensory state (e.g., 45° movement in Figure 2) exactly corresponds to the meaning conveyed by the encoded internal state.
In Choe and Smith (Reference Choe, Smith, Gil and Mooney2006), a visuomotor learning system was presented with the problem of grounding a set of visual states on its motor system, when the meaning of the visual states and motor primitives were initially unknown. A useful criterion is maintaining sensory invariance while acting. The sensory invariance criterion defines a probabilistic mapping from sensory states to motor primitives. In this experiment, some motor primitives maintained sensory invariance (e.g., by moving the agent parallel with the current sensory pattern), while other motor primitives violated sensory invariance (e.g., by moving perpendicular to the current sensory pattern). The agent learned to maintain sensory invariance while viewing a scene (see Fig. 2), which resulted in actions that moved the agent's visual field parallel to the current visual stimulus. The learned meaning of each sensory state was the set of motor primitives that resulted in maintaining that sensory state. Learning was accomplished with a standard reinforcement learning algorithm, and produced a mapping from visual states to congruent motor primitives (see Fig. 2). This pairing between sensory and motor primitives grounds sensory representations using action. The meaning of a sensory state is given by the actions that maintain it, and this meaning can be learned through interactions with the environment.
This approach can be extended to external sensory stimuli, such as the smiling face of a conspecific. When viewing a non-prototypical smile, a perceiver (visuomotor learner) can interact with the smile presenter, with the objective of maintaining the type of smile on the smile presenter's face. For example, when a perceiver sees an enjoyment smile, the objective is to maintain the enjoyment smile (i.e., sensory invariance). By interacting with the presenter (e.g., telling a joke, duplicating the perceived smile, or frowning), the perceiver can learn a mapping from the particular smile and his/her own motor behavior that are likely to maintain the smile type. Like the mapping shown in Figure 2, the meaning of a non-prototypical smile can be grounded by the actions (of the perceiver) that maintain it (in the presenter).
Unlike the computational experiment from Choe and Smith (Reference Choe, Smith, Gil and Mooney2006), where the initial mapping is randomly generated, some meaningful structure for smile interpretation may already be innate or learned from demonstrations of other people interacting in response to smiles. For example, prototypical smiles may be innately grounded, increasing likelihood of certain interactions when those smiles are observed. This prior knowledge about the range of interactions with a smile presenter could help reduce the search space needed to learn the meaning of non-prototypical smiles.
While the target article explains that the meanings of smiles are grounded in embodied cognition, the authors are not specific about which sensory-motor systems ground the meanings of smiles. We propose that, although some meaning can be captured by the facial sensory-motor system, important aspects of smile meaning cannot be captured without grounding smiles with respect to high-level actions making up the behavioral context. We have also described how a simple criterion (i.e., maintaining sensory invariance) can be used to ground the meanings of smiles on high-level actions.
You have
Access
Niedenthal et al. define embodied simulation as “a simulation of a state in the motor, somatosensory, affective, and reward systems that represents the meaning of the expression to the perceiver” (sect. 1, para. 4, italics in the original). In this sense, the state of the sensory-motor system classifies the smile (“represents the meaning”), but the meaning of the classification to the perceiver must also be grounded. Although prototypical facial expressions may be grounded in intrinsic reward systems, the subtleties of non-prototypical smiles likely must be learned and, therefore, may be grounded in the socio-behavioral context surrounding the smile: Smiles are not presented or perceived in isolation; they are surrounded by rich socio-behavioral context. For example, it is difficult to consider the meaning of dominance smiles without reference to more than facial expressions.
Like other forms of communication, smiles can have profound indirect consequences. For example, Sharlemann et al. (Reference Scharlemann, Eckel, Kacelnik and Wilson2001) investigated how smiles can be used to negotiate cooperation between humans. Smiles (and, more generally, facial expressions) provide clues about the values and objectives of other agents. Because smiles have only negligible direct physical impact, the meaning of a smile cannot be fully realized by considering only facial sensory-motor systems. Instead, we propose that an important component of the meaning of a smile must be grounded in high-level behaviors. But this raises a question: How can the meanings of smiles be grounded in high-level behaviors?
Choe and Smith (Reference Choe, Smith, Gil and Mooney2006) present a framework that explains the process of learning the meanings of internal sensory states by grounding them in terms of specific patterns of motor action, actions that maintain invariance in the internal sensory state. Our Figure 1 shows the framework, and Figure 2 shows how invariance in internal state can link up with motor-based meaning. The action that maintains invariance in the agent's internal sensory state (e.g., 45° movement in Figure 2) exactly corresponds to the meaning conveyed by the encoded internal state.
Visuomotor agent–its internal sensory state and motor primitives. Adapted from Choe and Smith (Reference Choe, Smith, Gil and Mooney2006).
Relationship between encoded sensory property and motor primitive giving internal state invariance. Adapted from Choe et al. (Reference Choe, Yang and Chern-Yeow Eng2007).
In Choe and Smith (Reference Choe, Smith, Gil and Mooney2006), a visuomotor learning system was presented with the problem of grounding a set of visual states on its motor system, when the meaning of the visual states and motor primitives were initially unknown. A useful criterion is maintaining sensory invariance while acting. The sensory invariance criterion defines a probabilistic mapping from sensory states to motor primitives. In this experiment, some motor primitives maintained sensory invariance (e.g., by moving the agent parallel with the current sensory pattern), while other motor primitives violated sensory invariance (e.g., by moving perpendicular to the current sensory pattern). The agent learned to maintain sensory invariance while viewing a scene (see Fig. 2), which resulted in actions that moved the agent's visual field parallel to the current visual stimulus. The learned meaning of each sensory state was the set of motor primitives that resulted in maintaining that sensory state. Learning was accomplished with a standard reinforcement learning algorithm, and produced a mapping from visual states to congruent motor primitives (see Fig. 2). This pairing between sensory and motor primitives grounds sensory representations using action. The meaning of a sensory state is given by the actions that maintain it, and this meaning can be learned through interactions with the environment.
This approach can be extended to external sensory stimuli, such as the smiling face of a conspecific. When viewing a non-prototypical smile, a perceiver (visuomotor learner) can interact with the smile presenter, with the objective of maintaining the type of smile on the smile presenter's face. For example, when a perceiver sees an enjoyment smile, the objective is to maintain the enjoyment smile (i.e., sensory invariance). By interacting with the presenter (e.g., telling a joke, duplicating the perceived smile, or frowning), the perceiver can learn a mapping from the particular smile and his/her own motor behavior that are likely to maintain the smile type. Like the mapping shown in Figure 2, the meaning of a non-prototypical smile can be grounded by the actions (of the perceiver) that maintain it (in the presenter).
Unlike the computational experiment from Choe and Smith (Reference Choe, Smith, Gil and Mooney2006), where the initial mapping is randomly generated, some meaningful structure for smile interpretation may already be innate or learned from demonstrations of other people interacting in response to smiles. For example, prototypical smiles may be innately grounded, increasing likelihood of certain interactions when those smiles are observed. This prior knowledge about the range of interactions with a smile presenter could help reduce the search space needed to learn the meaning of non-prototypical smiles.
While the target article explains that the meanings of smiles are grounded in embodied cognition, the authors are not specific about which sensory-motor systems ground the meanings of smiles. We propose that, although some meaning can be captured by the facial sensory-motor system, important aspects of smile meaning cannot be captured without grounding smiles with respect to high-level actions making up the behavioral context. We have also described how a simple criterion (i.e., maintaining sensory invariance) can be used to ground the meanings of smiles on high-level actions.