Book contents
- Pointing
- Pointing
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Preface
- Introduction to Pointing:
- Part One Culture and Development
- 1 Pointing at, Talking about, and Looking Away:
- 2 The Design of Pointing
- 3 Development of Pointing in Toddlers:
- 4 Attention Management by Deictic Gestures and Demonstratives:
- 5 Pointing to Dinner Participants in Deaf and Hearing Children’s and Adults’ Signing and Gesturing
- 6 The Cultural Nature of Early Social Communication
- 7 Pointing at Local Knowledge:
- 8 The Role of Infant Pointing in Child-Led Caregiving:
- 9 How Infants Learn to Follow Attention from Gaze-Direction and Pointing Gestures
- 10 Touch, Pointing and Mind-Reading
- 11 Socially Signaling Robotics:
- 12 Developmental Significance of the Human Pointing Gesture
- Part Two Evolutionary Perspectives
- Index
- References
11 - Socially Signaling Robotics:
A Novel Way of Looking at Gaze Following and Pointing
from Part One - Culture and Development
Published online by Cambridge University Press: 23 June 2026
Book contents
- Pointing
- Pointing
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Preface
- Introduction to Pointing:
- Part One Culture and Development
- 1 Pointing at, Talking about, and Looking Away:
- 2 The Design of Pointing
- 3 Development of Pointing in Toddlers:
- 4 Attention Management by Deictic Gestures and Demonstratives:
- 5 Pointing to Dinner Participants in Deaf and Hearing Children’s and Adults’ Signing and Gesturing
- 6 The Cultural Nature of Early Social Communication
- 7 Pointing at Local Knowledge:
- 8 The Role of Infant Pointing in Child-Led Caregiving:
- 9 How Infants Learn to Follow Attention from Gaze-Direction and Pointing Gestures
- 10 Touch, Pointing and Mind-Reading
- 11 Socially Signaling Robotics:
- 12 Developmental Significance of the Human Pointing Gesture
- Part Two Evolutionary Perspectives
- Index
- References
Summary
Language is the primary mode of communication for humans. However, nonverbal communication, such as gestures, eye contact, and facial expressions, also play a crucial role. In some cases, these nonverbal forms are even more important than spoken language. For example, pointing is a social behavior observed from early infancy and is documented not only in humans but also in other primates like chimpanzees. This chapter explores how Artificial Intelligence can offer a unique perspective on pointing.
Information
- Type
- Chapter
- Information
- PointingCulture, Development, and Evolution, pp. 211 - 223Publisher: Cambridge University PressPrint publication year: 2026
References
Ahmed, E., Buruk, O. O., & Hamari, J. (2024). Human–robot companionship: Current trends and future agenda. International Journal of Social Robotics, 16(8), 1809–1860.CrossRefGoogle Scholar
Baldwin, D. A. (1993). Early referential understanding: Infants’ ability to recognize referential acts for what they are. Developmental Psychology, 29(5), 832.CrossRefGoogle Scholar
Baldwin, D. A. (1995). Understanding the link between joint attention and language. In Moore, C. & Dunham, P. J. (Eds.), Joint attention: Its origins and role in development (pp. 131–158). Lawrence Erlbaum Associates, Inc.Google Scholar
Barwise, J. (1983). Information and semantics. Behavioral and Brain Sciences, 6(1), 65–65.CrossRefGoogle Scholar
Bayliss, A. P., Paul, M. A., Cannon, P. R., & Tipper, S. P. (2006). Gaze cuing and affective judgments of objects: I like what you look at. Psychonomic Bulletin & Review, 13, 1061–1066. https://doi.org/10.3758/BF03213926CrossRefGoogle ScholarPubMed
Bayliss, A. P., Frischen, A., Fenske, M. J., & Tipper, S. P. (2007). Affective evaluations of objects are influenced by observed gaze direction and emotional expression. Cognition, 104, 644–653. https://doi.org/10.1016/j.cognition.2006.07.012CrossRefGoogle ScholarPubMed
Becchio, C., Bertone, C., & Castiello, U. (2008). How the gaze of others influences object processing. Trends in Cognitive Sciences, 12, 254–258.CrossRefGoogle ScholarPubMed
Breazeal, C., & Scasselati, B. (1999). A context-dependent attention system for a social robot, in Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, 1146–1151.Google Scholar
Brinck, I. (2004). The pragmatics of imperative and declarative pointing. Cognitive Science Quarterly, 3(4), 429–446.Google Scholar
Carpenter, M., Nagell, K., & Tomasello, M. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monographs of the Society for Research in Child Development, 63(4), Ser. No. 255. https://doi.org/10.2307/1166214CrossRefGoogle ScholarPubMed
Cassell, J., Vilhjálmsson, H., & Bickmore, T. (2001). “BEAT: the Behavior Expression Animation Toolkit,” Proceedings of SIGGRAPH ‘01, pp. 477–486.CrossRefGoogle Scholar
Csibra, G., & Gergely, G. (2006). Social learning and social cognition: The case for pedagogy. Processes of change in brain and cognitive development. Attention and Performance XXI, 21, 249–274.CrossRefGoogle Scholar
Csibra, G., & Gergely, G. (2009). Natural pedagogy. Trends in Cognitive Sciences, 13(4), 148–153.CrossRefGoogle ScholarPubMed
Farroni, T., Johnson, M. H., Menon, E., Zulian, L., Faraguna, D., & Csibra, G. (2005). Newborns’ preference for face-relevant stimuli: Effects of contrast polarity. Proceedings of the National Academy of Sciences, 102, 17245–17250.CrossRefGoogle ScholarPubMed
Flom, R., Lee, K., & Muir, D. (2007). Gaze-following: Its development and significance. Lawrence Erlbaum Associates, Inc.Google Scholar
Gredebäck, G., Theuring, C., Hauf, P., & Kenward, B. (2008). The microstructure of infants’ gaze as they view adult shifts in overt attention. Infancy, 13, 533–543.CrossRefGoogle Scholar
Heylen, D., Kopp, S., Marsella, S. C., Pelachaud, C., & Vilhjálmsson, H. (2007). The next step towards a function markup language, Proceedings of the 7th international conference on Intelligent Virtual Agents, Table of Contents, pp. 270–280.Google Scholar
Imai, M., Ono, T., & Ishiguro, H. (2003). Physical relation and expression: Joint attention for human–robot interaction. IEEE Transactions on Industrial Electronics, 50, 636–643.CrossRefGoogle Scholar
Itakura, S., Ishida, H., Kanda, T., Shimada, Y., Ishiguro, H., & Lee, K. (2008). How to build an intentional android: Infants’ imitation of a robot’s goal-directed actions. Infancy, 13(5), 519–532.CrossRefGoogle Scholar
Johnson, M. H., Dziurawiec, S., Ellis, H., & Morton, J. (1991). Newborns’ preferential tracking of face-like stimuli and its subsequent decline. Cognition, 40, 1–19.CrossRefGoogle ScholarPubMed
Johnson, S., Slaughter, V., & Carey, S. (1998). Whose gaze will infants follow? The elicitation of gaze following in 12-month-olds. Developmental Science, 1, 233–238.CrossRefGoogle Scholar
Kanakogi, Y., & Itakura, S. (2011). Developmental correspondence between action prediction and motor ability in early infancy. Nature Communications, 2, 341. http://dx.doi.org/10.1038/ncomms1342CrossRefGoogle ScholarPubMed
Kanda, T., Sato, R., Saiwaki, N., & Ishiguro, H. (2007). A two-month field trial in an elementary school for long-term human–robot interaction. IEEE Transactions on Robotics, 23 (5), 962–971. https://doi.org/10.1109/TRO.2007.904904CrossRefGoogle Scholar
Kramer, J., & Scheutz, M. (2006, October). ADE: A framework for robust complex robotic architectures. 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4576–4581).CrossRefGoogle Scholar
Kranstedt, A., Kopp, S., & Wachsmuth, I. (2002). Murml: A multimodal utterance representation markup language for conversational agents. AAMAS’02 Workshop, Embodied conversational agents – Let’s specify and evaluate them! www.vhml.org/workshops/AAMAS/Google Scholar
Krause, M. A., Udell, M. A. R., Leavens, D. A., & Skopos, L. (2018). Animal pointing: Changing trends and findings from 30 years of research. Journal of Comparative Psychology, 132, 326–345. http://dx.doi.org/10.1037/com0000125CrossRefGoogle ScholarPubMed
Leung, E. H. L., & Rheingold, H. L. (1981). Development of pointing as a social gesture. Developmental Psychology, 17, 215–220. https://doi.org/10.1037/0012-1649.17.2.215CrossRefGoogle Scholar
Malnatsky, E., Wang, S., Hindriks, K. V., & Ligthart, M. E. (2025). Dialogic learning in child-robot interaction: A hybrid approach to personalized educational content generation. arXiv preprint, arXiv:2503.15762.Google Scholar
Meltzoff, A. N., Kuhl, P. K., Movellan, J., & Sejnowski, T. J. (2009). Foundations for a new science of learning. Science, 325, 284–288.CrossRefGoogle ScholarPubMed
Moubayed, S., Baklouti, M., Chetouani, M., Doutoit, T., Mahdhaoui, A., Martin, J.-C., Ondas, S., Pelachaud, C., Urbain, J., & Yilmaz, M. (2009). Generating robot/agent backchannels during a storytelling experiment, IEEE International Conference on Robotics and Automation (ICRA’09), pp. 3749–3754.Google Scholar
Mumme, D. L., & Fernald, A. (2003). The infant as onlooker: Learning from emotional reactions observed in a television scenario. Child Development, 74, 221–237. https://doi.org/10.1111/1467-8624.00532CrossRefGoogle Scholar
O’Connell, L., Poulin-Dubois, D., Demke, T., & Guay, A. (2009). Can infants use a nonhuman agent’s gaze direction to establish word–object relations? Infancy, 14, 414–438.CrossRefGoogle ScholarPubMed
Okumura, Y., Kanakogi, Y., Kanda, T., Ishiguro, H., & Itakura, S. (2013a). The power of human gaze on infant learning. Cognition, 128(2), 127–133.CrossRefGoogle ScholarPubMed
Okumura, Y., Kanakogi, Y., Kanda, T., Ishiguro, H., & Itakura, S. (2013b). Can infants use robot gaze for object learning?: The effect of verbalization. Interaction Studies, 14(3), 351–365. https://doi.org/10.1075/is.14.3.03okuCrossRefGoogle Scholar
Reid, V. M., & Striano, T. (2005). Adult gaze influences infant attention and object processing: Implications for cognitive neuroscience. European Journal of Neuroscience, 21, 1763–1766.CrossRefGoogle ScholarPubMed
Repacholi, B. M. (1998). Infants’ use of attentional cues to identify the referent of another person’s emotional expression. Developmental Psychology, 34(5), 1017.CrossRefGoogle ScholarPubMed
Salma, Z., Hijón-Neira, R., Pizarro, C., & Abdul Moqeet, A. (2025). Effectiveness of robot-mediated learning in fostering children’s social and cognitive development. Applied Sciences, 15(7), 3567.CrossRefGoogle Scholar
Satake, S., Hayashi, K., Nakatani, K., & Kanda, T. (2015, September). Field trial of an information-providing robot in a shopping mall. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1832–1839). IEEE.CrossRefGoogle Scholar
Senju, A., & Csibra, G. (2008). Gaze following in human infants depends on communicative signals. Current Biology, 18, 668–671.CrossRefGoogle Scholar
Shi, C., Kanda, T., Shimada, M., Yamaoka, F., Ishiguro, H., & Hagita, N. (2010, October). Easy development of communicative behaviors in social robots. In 2010 IEEE/RSJ international conference on intelligent robots and systems (pp. 5302–5309). IEEE.Google Scholar
Shiwa, T., Kanda, T., Imai, M., Ishiguro, H., & Hagita, N. (2008). How quickly should communication robots respond? Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (pp. 153–160). https://doi.org/10.1145/1349822.1349843CrossRefGoogle Scholar
Southgate, V., Van Maanen, C., & Csibra, G. (2007). Infant pointing: Communication to cooperate or communication to learn? Child Development, 78(3), 735–740.CrossRefGoogle ScholarPubMed
Sperber, D., & Wilson, D. (1995). Relevance: Communication and cognition (2nd ed.). Wiley: Blackwell.Google Scholar
Theuring, C., Gredeback, G., & Hauf, P. (2007). Object processing during a joint gaze following task. European Journal of Developmental Psychology, 4, 65–79.CrossRefGoogle Scholar
Tomasello, M. (1995). Joint attention as social cognition. In Moore, C. & Dunham, P. J. (Eds.), Joint attention: Its origins and role in development (pp. 103–130). Lawrence Erlbaum Associates.Google Scholar
Tomasello, M. (1999). The cultural origins of human cognition. Harvard University Press.Google Scholar
Tomasello, M., Carpenter, M., & Liszkowski, U. (2007). A new look at infant pointing. Child Development, 78, 705–722. https://doi.org/10.1111/j.1467-8624.2007.01025.xCrossRefGoogle Scholar
Topal, J., Gergely, G., Miklosi, A., Erdohegyi, A., & Csibra, G. (2008). Infants’ perseverative search errors are induced by pragmatic misinterpretation. Science, 321, 1831–1834.CrossRefGoogle ScholarPubMed
Trafton, J. G., Bugajska, M. D., Fransen, B. R., & Ratwani, R. M. (2008). Integrating vision and audition within a cognitive architecture to track conversations, ACM/IEEE Conference on Human–Robot Interaction (HRI2008) (pp. 201–208).Google Scholar
Vilhjalmsson, H., Cantelmo, N., Cassell, J., Chafai, N., Kipp, M., Kopp, S., Mancini, M., Marsella, S., Marshall, A. N., Pelachaud, C., Ruttkay, Z., Thórisson, K. R., van Welbergen, H., & van der Werf, R. J. (2007). The Behavior Markup Language: Recent developments and challenges, Proceedings of the 7th international conference on Intelligent Virtual Agents, table of contents (pp. 99–111).Google Scholar
Woodward, A. L. (1998). Infants selectively encode the goal object of an actor’s reach. Cognition, 69, 1–34.CrossRefGoogle ScholarPubMed
Woodward, A. L. (2003). Infants’ developing understanding of the link between looker and object. Developmental Science, 6(3), 297–311.CrossRefGoogle Scholar
Yamamoto, S., Tetsui, T., Naganuma, M., & Kimura, R. (2006, October). Trial of using robotic pet as human interface of multimedia education system for pre-school aged child in kindergarten. 2006 SICE–ICASE International Joint Conference (pp. 3398–3403).CrossRefGoogle Scholar
Yamaoka, F., Kanda, T., Ishiguro, H., & Hagita, N. (2009). Developing a model of robot behavior to identify and appropriately respond to implicit attention-shifting. Proceedings of the 4th ACM/IEEE international conference on Human robot interaction (HRI ‘09), 133–140. Association for Computing Machinery. https://doi.org/10.1145/1514095.1514120Google Scholar
Yamazaki, A., Yamazaki, K., Kuno, Y., Burdelski, M., Kawashima, M., & Kuzuoka, H. (2008, April). Precision timing in human–robot interaction: Coordination of head movement and utterance. Proceedings of the SIGCHI conference on human factors in computing systems (pp. 131–140).CrossRefGoogle Scholar
Yoon, J. M. D., Johnson, M. H., & Csibra, G. (2008). Communication-induced memory biases in preverbal infants. Proceedings of the National Academy of Sciences, 105, 13690–13695.CrossRefGoogle ScholarPubMed
Accessibility standard: WCAG 2.2 AAA
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
The HTML of this chapter
complies with version 2.2 of the
Web Content Accessibility Guidelines
(WCAG), offering more comprehensive accessibility measures for a broad range
of users and attains the highest
(AAA)
level of WCAG compliance, optimising the user experience by meeting the most extensive
accessibility guidelines.
Content Navigation
Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Reading Order & Textual Equivalents
Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
Visual Accessibility
Use of colour is not sole means of conveying information
You will still understand key ideas or prompts without relying solely on colour, which is especially helpful if you have colour vision deficiencies.
You will still understand key ideas or prompts without relying solely on colour, which is especially helpful if you have colour vision deficiencies.
Structural and Technical Features
ARIA roles provided
You gain clarity from ARIA (Accessible Rich Internet Applications) roles and attributes, as they help assistive technologies interpret how each part of the content functions.
You gain clarity from ARIA (Accessible Rich Internet Applications) roles and attributes, as they help assistive technologies interpret how each part of the content functions.