Book contents
3 - Can AI Be Psychologically Safe?
Published online by Cambridge University Press: 24 July 2026
Summary
This chapter explores the psychological impacts of automation that have received insufficient attention in safety frameworks. We examine how AI safety extends beyond technical robustness to encompass protection from psychological harms including manipulation, over-reliance, and exploitation of vulnerability. The chapter presents established methodologies for addressing psychological safety risks and identifies critical gaps in current approaches. We argue that comprehensive AI safety requires not only technical safeguards but also robust frameworks for protecting human agency, autonomy, and wellbeing. As AI systems become more integrated into everyday life in more sensitive ways, including companions and advanced assistants, psychological safety considerations become essential for responsible development. We conclude by outlining key priorities for further research and contend that a critical window of opportunity exists to establish effective guardrails. As we develop these technologies, they are simultaneously shaping us in ways that may prove undesirable and difficult to reverse.
Information
- Type
- Chapter
- Information
- Operationalizing Responsible AI , pp. 42 - 62Publisher: Cambridge University PressPrint publication year: 2026
References
Abdelwanis, M., Alarafati, H. K., Tammam, M. M. S., and Simsekler, M. C. E. 2024. Exploring the risks of automation bias in healthcare artificial intelligence applications: A Bowtie analysis. Journal of Safety Science and Resilience, 5(4), 460–469.CrossRefGoogle Scholar
Amershi, S., et al. 2019. Guidelines for human–AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13.Google Scholar
Bhat, R., Kowshik, S., Suresh, S., Alamelu, G., Gite, S., and Albattat, A. 2025. Digital companionship or psychological risk? The role of AI characters in shaping youth mental health. Asian Journal of Psychiatry, 104, article 104356.CrossRefGoogle ScholarPubMed
Blut, M., Wang, C., Wünderlich, N. V., and Brock, C. 2021. Understanding anthropomorphism in service provision: A meta-analysis of physical robots, chatbots, and other AI. Journal of the Academy of Marketing Science, 49, 632–658.CrossRefGoogle Scholar
Bogle, M. 2024. Character.AI users still seeing explicit, violent companionship. Carolina News and Reporter, December 11. https://carolinanewsandreporter.cic.sc.edu/character-ai-users-still-seeing-explicit-violent-companionship/ (last accessed February 10, 2026).Google Scholar
Boston Dynamics. 2022. General purpose robots should not be weaponized. www.bostondynamics.com/open-letter-opposing-weaponization-general-purpose-robots (last accessed February 17, 2026).Google Scholar
Buolamwini, J., and Gebru, T. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15.Google Scholar
Burwell, S., Sample, M., and Racine, E. 2017. Ethical aspects of brain computer interfaces: a scoping review. BMC Medical Ethics, 18, article 60.CrossRefGoogle Scholar
Burton, J. W., Stein, M. K., and Jensen, T. B. 2020a. A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239.CrossRefGoogle Scholar
Burton, S., Habli, I., Lawton, T., McDermid, J., Morgan, P., and Porter, Z. 2020b. Mind the gaps: Assuring the safety of autonomous systems from an engineering, ethical, and legal perspective. Artificial Intelligence, 279, 103201.CrossRefGoogle Scholar
Darling, K. 2021. The New Breed: What Our History with Animals Reveals about Our Future with Robots (New York: Henry Holt and Company).Google Scholar
European Commission. 2021. Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence. COM(2021) 206 final (Brussels: European Commission).Google Scholar
Folk, D. P., Wu, C., and Heine, S. J. 2025. Cultural variation in attitudes toward social chatbots. Journal of Cross-Cultural Psychology, 56(3).CrossRefGoogle ScholarPubMed
Gabriel, I., Manzini, A., Keeling, G., Hendricks, L. A., Rieser, V., Iqbal, H., et al. 2024. The ethics of advanced AI assistants. arXiv preprint arXiv:2404.16244.Google Scholar
The Guardian. 2022a. Google fires software engineer who claims AI chatbot is sentient. The Guardian, July 23. www.theguardian.com/technology/2022/jul/23/google-fires-software-engineer-who-claims-ai-chatbot-is-sentient (last accessed February 10, 2026).Google Scholar
The Guardian. 2022b. Chess robot grabs and breaks finger of seven-year-old opponent. The Guardian, July 24. www.theguardian.com/sport/2022/jul/24/chess-robot-grabs-and-breaks-finger-of-seven-year-old-opponent-moscow/ (last accessed February 19, 2026).Google Scholar
Lalwani, H. and Salam, H. 2026. The Supportiveness-Safety Tradeoff in LLM well-being agents. arXiv preprint arXiv:2602.04487.CrossRefGoogle Scholar
Liao, Q. V., Gruen, D., and Miller, S. 2020. Questioning the AI: Informing design practices for explainable AI user experiences. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15.Google Scholar
Logan, D. E., Breazeal, C., Goodwin, M. S., Jeong, S., O’Connell, B., Smith-Freedman, D., Heathers, J., and Weinstock, P. 2019. Social robots for hospitalized children. Pediatrics, 143(6), e20181511Google Scholar
Lopez, J. A., et al. 2024. Privacy-preserving optics for enhancing protection in face de-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–10.Google Scholar
MIT Media Lab and OpenAI. 2025. Early methods for studying affective use and emotional wellbeing in ChatGPT: An OpenAI and MIT Media Lab Research collaboration. MIT Media Lab, March 21. www.media.mit.edu/posts/openai-mit-research-collaboration-affective-use-and-emotional-wellbeing-in-ChatGPT/ (last accessed February 10, 2026).Google Scholar
ML Commons. 2024. Benchmark Work, MLCommons. https://mlcommons.org/benchmarks/ (last accessed February 17, 2026).Google Scholar
oxethica. 2024. AI governance platform: Simplify compliance. oxethica. www.oxethica.com/ (last accessed February 17, 2026).Google Scholar
Parasuraman, R., and Manzey, D. H. 2010. Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410.CrossRefGoogle Scholar
Reuters. 2023. Italy bans US-based AI chatbot Replika from using personal data. Reuters, February 3. www.reuters.com/technology/italy-bans-us-based-ai-chatbot-replika-using-personal-data-2023-02-03/ (last accessed February 17, 2026).Google Scholar
Schaefer, K. E., Chen, J. Y. C., Szalma, J. L., and Hancock, P. A. 2016. A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58(3).CrossRefGoogle ScholarPubMed
van der Schyff, E. L., Ridout, B., Amon, K. L., Forsyth, R., and Campbell, A. J. 2023. Providing mental health support through an AI-powered chatbot (Leora): Protocol for a randomized controlled trial. JMIR Research Protocols, 12, e46448.Google Scholar
Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E., and DeMarco, J. 2023. The psychological impacts of content moderation on content moderators: A qualitative study. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 17(4), article 8.CrossRefGoogle Scholar
Steenstra, I., Pedrelli, P., Shi, W., Marsella, S., and Bickmore, T. W. 2026. Assessing risks of large language models in mental health support: A framework for automated clinical AI red teaming. arXiv preprint arXiv:2602.19948.Google Scholar
Talbott, S. J. 2025. The Waymo vandalism incident: A wake-up call for the AV industry’, Forbes, January 27. www.forbes.com/sites/selikajosiahtalbott/2025/01/27/the-waymovandalism-incident-a-wake-up-call-for-the-av-industry/ (last accessed April 18, 2025).Google Scholar
Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., Bergman, S., Kay, J., Griffin, C., Bariach, B., Gabriel, I., Rieser, V. and Isaac, W. (2023) ‘Sociotechnical Safety Evaluation of Generative AI Systems’, arXiv preprint arXiv:2310.11986.Google Scholar
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems (NeurIPS), 35, 24824–24837.Google Scholar
Yuste, R., Goering, S., Arcas, B. A. Y., Bi, G., Carmena, J. M., Carter, A., et al. 2017. Four ethical priorities for neurotechnologies and AI. Nature, 551(7679), 159–163.CrossRefGoogle ScholarPubMed
Accessibility standard: WCAG 2.2 AAA
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
The HTML of this chapter complies with version 2.2 of the
Web Content Accessibility Guidelines
(WCAG), offering more comprehensive accessibility measures for a broad range
of users and attains the highest
(AAA)
level of WCAG compliance, optimising the user experience by meeting the most extensive
accessibility guidelines.
Content Navigation
Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Reading Order & Textual Equivalents
Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
Visual Accessibility
Use of high contrast between text and background colour
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.