How Successful Are Existing Algorithms in Explaining and Predicting Human Behaviour?

doi:10.1017/9781108939010.017

12 - How Successful Are Existing Algorithms in Explaining and Predicting Human Behaviour?

from Part III - Algorithmic Behaviour

Published online by Cambridge University Press: 29 May 2026

Alexander Kharlamov

Edited by

Ganna Pogrebna and

Thomas T. Hills

Show author details

Ganna Pogrebna: Affiliation:
The Alan Turing Institute
Thomas T. Hills: Affiliation:
University of Warwick

Book contents

Get access

Summary

This chapter critically reviews the success of existing algorithms in explaining and predicting human behaviour. While traditional statistical methods have limitations in this area, algorithmic approaches have gained popularity. The chapter covers a range of algorithms, including decision trees, neural networks and clustering algorithms, evaluating their strengths and limitations in various applications. It also considers ethical concerns, such as bias and privacy violations and the need for transparency and explainability. The chapter emphasises the importance of interdisciplinary collaboration between computer science, statistics and behavioural science, and the need for ongoing development and refinement of these methods. By evaluating the effectiveness of algorithmic approaches to human behaviour, this chapter is a valuable resource for researchers and practitioners in the field.

Keywords

algorithms behavioural data science decision trees neural networks clustering algorithms predictive analysis bias privacy ethics

Information

Type: Chapter
Information: The Cambridge Handbook of Behavioural Data Science , pp. 204 - 222
DOI: https://doi.org/10.1017/9781108939010.017 [Opens in a new window]
Publisher: Cambridge University Press
Print publication year: 2026

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Ananny, M. (2016). Toward an ethics of algorithms: Convening, observation, probability, and timeliness. Science, Technology, & Human Values, 41(1), 93–117. https://doi.org/10.1177/0162243915606523.CrossRef Google Scholar

Andrejevic, M. & Gates, K. (2014). Big data surveillance: Introduction. Surveillance & Society, 12(2), 185–196. https://doi.org/10.24908/ss.v12i2.5113.CrossRef Google Scholar

Angwin, J., Larson, J., Mattu, S. & Kirchner, L. (2016). Machine bias. ProPublica. www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.Google Scholar

Barocas, S., Hardt, M. & Narayanan, A. (2023). Fairness and machine learning: Limitations and opportunities. MIT Press.Google Scholar

Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big?, in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, virtual conference, 3–10 March, pp. 610–623. https://doi.org/10.1145/3442188.3445922.CrossRef Google Scholar

Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77–84. https://doi.org/10.1145/2133806.2133826.CrossRef Google Scholar

Bzdok, D., Altman, N. & Krzywinski, M. (2018). Statistics versus machine learning. Nature Methods, 15(4), 233–234. https://doi.org/10.1038/nmeth.4642.CrossRef Google Scholar PubMed

Can, Y. S., Arnrich, B. & Ersoy, C. (2019). Stress detection in daily life scenarios using smart phones and wearable sensors: A survey. Journal of Biomedical Informatics, 92, 103139. https://doi.org/10.1016/j.jbi.2019.103139.CrossRef Google Scholar PubMed

Corbett-Davies, S. & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023.Google Scholar

Dolnicar, S. & Grün, B. (2008). Challenging ‘factor–cluster segmentation’. Journal of Travel Research, 47(1), 63–71. https://doi.org/10.1177/0047287508318910.CrossRef Google Scholar

Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78–87. https://doi.org/10.1145/2347736.2347755.CrossRef Google Scholar

Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.Google Scholar

Eubanks, V. (2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.Google Scholar

Garcez, A. D., Lamb, L. C. & Gabbay, D. M. (2012). Neural–symbolic cognitive reasoning. Springer.Google Scholar

Gigerenzer, G. & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482. https://doi.org/10.1146/annurev-psych-120709-145346.CrossRef Google Scholar PubMed

Griffiths, T. L., Chater, N., Kemp, C., Perfors, A. & Tenenbaum, J. B. (2008). Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 12(11), 394–403. https://doi.org/10.1016/j.tics.2008.07.006.Google Scholar

Grootendorst, M. (2022). BERTopic: Neural topic modeling with class-based TF-IDF. arXiv preprint arXiv:2203.05794.Google Scholar

Helberger, N., Eskens, S. & Möller, J. (2020). Challenging the algorithms: Accountability for the use of algorithms in the public sector. Journal of Information, Communication and Ethics in Society, 18(2), 173–188. https://doi.org/10.1108/JICES-10-2019-0121.Google Scholar

Henrich, J., Heine, S. J. & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X.CrossRef Google Scholar PubMed

Hochreiter, S. & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.CrossRef Google Scholar PubMed

Huang, S., Li, R., Liu, K. & Ma, X. (2022). Understanding vaccination uptake via behavioural and computational models. Journal of Biomedical Informatics, 125, 103976. https://doi.org/10.1016/j.jbi.2021.103976.Google Scholar

van der Laan, M. J. & Rose, S. (2011). Targeted learning: Causal inference for observational and experimental data. Springer.10.1007/978-1-4419-9782-1CrossRef Google Scholar

LeCun, Y., Bengio, Y. & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539.CrossRef Google Scholar PubMed

Lemmens, A. & Croux, C. (2006). Bagging and boosting classification trees to predict churn. Journal of Marketing Research, 43(2), 276–286. https://doi.org/10.1509/jmkr.43.2.276.CrossRef Google Scholar

Lipton, Z. C. (2018). The mythos of model interpretability. Queue, 16(3), 31–57. https://doi.org/10.1145/3236386.3241340.CrossRef Google Scholar

Lo, A. (2017). Adaptive markets: Financial evolution at the speed of thought. Princeton University Press.Google Scholar

Lundberg, S. M. & Lee, S. I. (2017). A unified approach to interpreting model predictions, in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, 4–9 December, 4765–4774.Google Scholar

Molnar, C. (2022). Interpretable machine learning: A guide for making black box models explainable (2nd ed.), GitHub. https://christophm.github.io/interpretable-ml-book/.Google Scholar

Mount, M. K., Barrick, M. R., Scullen, S. M. & Rounds, J. (2005). Higher‐order dimensions of the big five personality traits and the big six vocational interest types. Personnel Psychology, 58(2), 447–478.10.1111/j.1744-6570.2005.00468.xCrossRef Google Scholar

Passi, S. & Barocas, S. (2019). Problem formulation and fairness, in Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, 29–31 January, 39–48. https://doi.org/10.1145/3287560.3287567.CrossRef Google Scholar

Pearl, J. (2009). Causality: Models, reasoning and inference (2nd ed.). Cambridge University Press.10.1017/CBO9780511803161CrossRef Google Scholar

Resnik, P., Armstrong, W., Claudino, L., Nguyen, T., Nguyen, V. A. & Boyd-Graber, J. (2015). Beyond LDA: Exploring supervised topic modeling for depression-related language in Twitter, in Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology, Denver, CO, 5 June, 99–107. https://doi.org/10.18653/v1/W15-1212.CrossRef Google Scholar

Ribeiro, M. T., Singh, S. & Guestrin, C. (2016). ‘Why should I trust you?’: Explaining the predictions of any classifier, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, 17 August, 1135–1144.10.1145/2939672.2939778Google Scholar

Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x.CrossRef Google Scholar PubMed

Shorten, C. & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1–48. https://doi.org/10.1186/s40537-019-0197-0.CrossRef Google Scholar

Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. https://doi.org/10.1214/10-STS330.CrossRef Google Scholar

Strubell, E., Ganesh, A. & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243.Google Scholar

Susser, D., Roessler, B. & Nissenbaum, H. (2019). Online manipulation: Hidden influences in a digital world. Georgetown Law Technology Review, 4, 1–45.Google Scholar

Sutton, R. S. & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.Google Scholar

Tversky, A. & Kahneman, D. (1991). Loss aversion in riskless choice: A reference-dependent model. The Quarterly Journal of Economics, 106(4), 1039–1061. https://doi.org/10.2307/2937956.CrossRef Google Scholar

Veinot, T. C., Mitchell, H. & Ancker, J. S. (2018). Good intentions are not enough: How informatics interventions can worsen inequality. Journal of the American Medical Informatics Association, 25(8), 1080–1088. https://doi.org/10.1093/jamia/ocy052.CrossRef Google Scholar

Wachter, S., Mittelstadt, B. & Floridi, L. (2017). Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. International Data Privacy Law, 7(2), 76–99. https://doi.org/10.1093/idpl/ipx005.CrossRef Google Scholar

Walsh, E. (2025). Does a face speak for itself? Emotion recognition technologies and explainable AI. Philosophy & Technology, 38(2), 67.10.1007/s13347-025-00891-8CrossRef Google Scholar

Watson, D. S., Krishnan, A. & Raghavan, M. (2023). Foundations of explainable machine learning. Annual Review of Statistics and Its Application, 10, 1–27. https://doi.org/10.1146/annurev-statistics-040220-113825.Google Scholar

Wu, Q., Zhang, D., Xu, G. & Lu, B. (2022). Automatic deception detection in videos using facial micro-expressions and deep learning. IEEE Transactions on Affective Computing, 13(2), 799–810. https://doi.org/10.1109/TAFFC.2020.2979844.Google Scholar

Yarkoni, T. (2020). The generalizability crisis. Behavioral and Brain Sciences, 45, e1. https://doi.org/10.1017/S0140525X20001685.CrossRef Google Scholar PubMed

Yarkoni, T. & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393.CrossRef Google Scholar PubMed

Yeung, K. (2017). ‘Hypernudge’: Big data as a mode of regulation by design. Information, Communication & Society, 20(1), 118–136. https://doi.org/10.1080/1369118X.2016.1186713.CrossRef Google Scholar

Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. PublicAffairs.Google Scholar

Accessibility standard: WCAG 2.2 AAA

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

The HTML of this chapter complies with version 2.2 of the Web Content Accessibility Guidelines (WCAG), offering more comprehensive accessibility measures for a broad range of users and attains the highest (AAA) level of WCAG compliance, optimising the user experience by meeting the most extensive accessibility guidelines.

Content Navigation

Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.

Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.

Reading Order & Textual Equivalents

Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.

Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.

Full alternative textual descriptions
You get more than just short alt text: you have comprehensive text equivalents, transcripts, captions, or audio descriptions for substantial non‐text content, which is especially helpful for complex visuals or multimedia.

Visualised data also available as non-graphical data
You can access graphs or charts in a text or tabular format, so you are not excluded if you cannot process visual displays.

Visual Accessibility

Use of colour is not sole means of conveying information
You will still understand key ideas or prompts without relying solely on colour, which is especially helpful if you have colour vision deficiencies.

Use of high contrast between text and background colour
You benefit from high‐contrast text, which improves legibility if you have low vision or if you are reading in less‐than‐ideal lighting conditions.