Book contents
- The Cambridge Handbook of Behavioural Data Science
- Cambridge Handbooks in Psychology
- Reviews
- The Cambridge Handbook of Behavioural Data Science
- Copyright page
- Dedication
- Contents
- Contributors
- Preface
- Introduction
- Part IIntroduction to Behavioural Data Science
- Part IIHuman Behaviour
- 4Behavioural Data Science for Understanding Human Decisions, Choices and Judgement
- 5Psychological Theories of Decision-Making under Risk
- 6Prediction-Oriented Behavioural Research and Its Relationship to Classical Decision Research
- 7The ABCs of Behavioural Influence
- 8Word and Sentence Embedding Methods for Studying Human Behaviour
- 9Predictive Bayesian Modelling in Cognitive Sciences
- 10Human Aspects of AI-Related Risks
- Part IIIAlgorithmic Behaviour
- Part IVSystems and Culture
- Part VApplications
- Part VIConcluding Remarks
- Main Abbreviations and Acronyms
- Glossary
- References
8 - Word and Sentence Embedding Methods for Studying Human Behaviour
from Part II - Human Behaviour
Published online by Cambridge University Press: 29 May 2026
Book contents
- The Cambridge Handbook of Behavioural Data Science
- Cambridge Handbooks in Psychology
- Reviews
- The Cambridge Handbook of Behavioural Data Science
- Copyright page
- Dedication
- Contents
- Contributors
- Preface
- Introduction
- Part IIntroduction to Behavioural Data Science
- Part IIHuman Behaviour
- 4Behavioural Data Science for Understanding Human Decisions, Choices and Judgement
- 5Psychological Theories of Decision-Making under Risk
- 6Prediction-Oriented Behavioural Research and Its Relationship to Classical Decision Research
- 7The ABCs of Behavioural Influence
- 8Word and Sentence Embedding Methods for Studying Human Behaviour
- 9Predictive Bayesian Modelling in Cognitive Sciences
- 10Human Aspects of AI-Related Risks
- Part IIIAlgorithmic Behaviour
- Part IVSystems and Culture
- Part VApplications
- Part VIConcluding Remarks
- Main Abbreviations and Acronyms
- Glossary
- References
Summary
This chapter examines distributed semantics and language models, Word2Vec and BERT, for obtaining word and sentence embeddings to extract knowledge representations from naturalistic stimuli in behavioural data science applications. These representations enable large-scale study of human cognition and behaviour, providing insights into higher-level psychological processes. Utilising distributed semantics is cost-effective due to pretrained language models’ availability. However, integrating perceptual data with semantic representations is crucial, as these representations are experience-based and may not fully capture complex sentences’ meaning. Future research can address questions like memorability using semantic representations and explore individual differences by training models on data from distinct sub-populations. High-dimensional representations from embedding models offer numerous opportunities to study human behaviour on a larger scale than previously possible.
Keywords
Information
- Type
- Chapter
- Information
- The Cambridge Handbook of Behavioural Data Science , pp. 142 - 156Publisher: Cambridge University PressPrint publication year: 2026
References
Aka, A. & Bhatia, S. (2022). Machine learning models for predicting, understanding and influencing health perception. Journal of the Association for Consumer Research, 7(2), 142–153.10.1086/718456CrossRefGoogle Scholar
Aka, A., Olivola, C., Bhatia, S., & Nave, G. (2020). Computational consumer segmentation and brand management. Advances in Consumer Research, 48, 825–830.Google Scholar
Atkinson, N., Saperstein, S. & Pleis, J. (2009). Using the internet for health-related activities: Findings from a national probability sample. Journal of Medical Internet Research, 11(1), e5.10.2196/jmir.1035CrossRefGoogle ScholarPubMed
Bhatia, S. (2017a). Associative judgment and vector space semantics. Psychological Review, 124(1), 1.10.1037/rev0000047CrossRefGoogle ScholarPubMed
Bhatia, S. (2017b). The semantic representation of prejudice and stereotypes. Cognition, 164, 46–60.10.1016/j.cognition.2017.03.016CrossRefGoogle ScholarPubMed
Bhatia, S. (2019). Predicting risk perception: New insights from data science. Management Science, 65(8), 3449–3947.10.1287/mnsc.2018.3121CrossRefGoogle Scholar
Bhatia, S. & Olivola, C. (2021). Computational brand perception: Word embedding techniques for predicting consumer brand-trait perceptions. Working paper.Google Scholar
Bhatia, S. & Walasek, L. (2019). Association and response accuracy in the wild. Memory & Cognition, 47(2), 292–298.10.3758/s13421-018-0869-6CrossRefGoogle Scholar
Buchanan, E. M., Valentine, K. D. & Maxwell, N. P. (2019). English semantic feature production norms: An extended database of 4436 concepts. Behavior Research Methods, 1–15. https://doi.org/10.3758/s13428-019-01243-zCrossRefGoogle Scholar
Caliskan, A., Bryson, J. J. & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186.10.1126/science.aal4230CrossRefGoogle ScholarPubMed
Devereux, B. J., Tyler, L. K., Geertzen, J. & Randall, B. (2014). The Centre for Speech, Language and the Brain (CSLB) concept property norms. Behavior Research Methods, 46(4), 1119–1127.10.3758/s13428-013-0420-4CrossRefGoogle ScholarPubMed
Devlin, J., Chang, M. W., Lee, K. & Toutanova, K. (2019, June). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American chapter of the association for computational linguistics: Human language technologies, Minneapolis, 2–7 June, volume 1 (long and short papers), 4171–4186.Google Scholar
Goldberg, L. R., Johnson, J. A., Eber, H. W., et al. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 84–96.10.1016/j.jrp.2005.08.007CrossRefGoogle Scholar
Griffiths, T. L., Steyvers, M. & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114(2), 211.10.1037/0033-295X.114.2.211CrossRefGoogle ScholarPubMed
Jones, M. N., Willits, J. A. & Dennis, S. (2015). Models of semantic memory, in Oxford handbook of computational and mathematical psychology, edited by Busemeyer, J. R., Wang, Z., Townsend, J. T. & Eidels, A.. Oxford University Press, 232–254.Google Scholar
Kosinski, M., Stillwell, D. & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15), 5802–5805.10.1073/pnas.1218772110CrossRefGoogle ScholarPubMed
Landauer, T. K. & Dumais, S (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.10.1037/0033-295X.104.2.211CrossRefGoogle Scholar
Lenci, A. (2018). Distributional models of word meaning. Annual Review of Linguistics, 4, 151–171.10.1146/annurev-linguistics-030514-125254CrossRefGoogle Scholar
Liu, X., He, P., Chen, W. & Gao, J. (2019). Multi-task deep neural networks for natural language understanding. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 28 July–2 August, 4487–4496.10.18653/v1/P19-1441CrossRefGoogle Scholar
Mandera, P., Keuleers, E. & Brysbaert, M. (2017). Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting: a review and empirical validation. Journal of Memory and Language, 92, 57–78.10.1016/j.jml.2016.04.001CrossRefGoogle Scholar
McRae, K., Cree, G. S., Seidenberg, M. S. & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37, 547–559.10.3758/BF03192726CrossRefGoogle ScholarPubMed
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S. & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26, 3111–3119.Google Scholar
Powell, J., Inglis, N., Ronnie, J. & Large, S. (2011). The characteristics and motivations of online health information seekers: Cross-sectional survey and qualitative interview study. Journal of Medical Internet Research, 13(1), e20.10.2196/jmir.1600CrossRefGoogle ScholarPubMed
Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. (2018). Improving language understanding by generative pre-training. Working paper, available at: www.mikecaptain.com/resources/pdf/GPT-1.pdf (last accessed 29 September 2025).Google Scholar
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D. & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9.Google Scholar
Rong, X. (2014). word2vec parameter learning explained. arXiv preprint arXiv:1411.2738.Google Scholar
Sanh, V., Debut, L., Chaumond, J. & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.Google Scholar
Shepard, R. N. (1974). Representation of structure in similarity data: problems and prospects. Psychometrika, 39, 373–421.10.1007/BF02291665CrossRefGoogle Scholar
Shepard, R. N. & Arabie, P. (1979). Additive clustering: Representation of similarities as combinations of discrete overlapping properties. Psychological Review, 86(2), 87.10.1037/0033-295X.86.2.87CrossRefGoogle Scholar
Tausczik, Y. R. & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24–54.10.1177/0261927X09351676CrossRefGoogle Scholar
Tversky, A. & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293.10.1037/0033-295X.90.4.293CrossRefGoogle Scholar
Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), edited by von Luxburg, U., Guyon, I. & Bengio, S., Long Beach, CA, 6000–6010.Google Scholar
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018). Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 13(3), 55–75.10.1109/MCI.2018.2840738CrossRefGoogle Scholar
Accessibility standard: WCAG 2.2 AAA
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
The HTML of this chapter complies with version 2.2 of the
Web Content Accessibility Guidelines
(WCAG), offering more comprehensive accessibility measures for a broad range
of users and attains the highest
(AAA)
level of WCAG compliance, optimising the user experience by meeting the most extensive
accessibility guidelines.
Content Navigation
Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Reading Order & Textual Equivalents
Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.
Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.
Visual Accessibility
Use of colour is not sole means of conveying information
You will still understand key ideas or prompts without relying solely on colour, which is especially helpful if you have colour vision deficiencies.
You will still understand key ideas or prompts without relying solely on colour, which is especially helpful if you have colour vision deficiencies.
Use of high contrast between text and background colour
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.