Hostname: page-component-74656f9884-rdgjk Total loading time: 0 Render date: 2026-11-09T11:09:57.002Z Has data issue: false hasContentIssue false
  • English
  • Français

Using response times to measure ability on a cognitive task

Published online by Cambridge University Press:  17 January 2025

Aleksandr Alekseev Open the ORCID record for Aleksandr Alekseev [Opens in a new window]
Aleksandr Alekseev*
Affiliation:
Economic Science Institute, Chapman University, One University Drive, 92866 Orange, CA, USA
*
e-mail: alekseev@chapman.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

I show how using response times as a proxy for effort can address a long-standing issue of how to separate the effect of cognitive ability on performance from the effect of motivation. My method is based on a dynamic stochastic model of optimal effort choice in which ability and motivation are the structural parameters. I show how to estimate these parameters from the data on outcomes and response times in a cognitive task. In a laboratory experiment, I find that performance on a digit-symbol test is a noisy and biased measure of cognitive ability. Ranking subjects by their performance leads to an incorrect ranking by their ability in a substantial number of cases. These results suggest that interpreting performance on a cognitive task as ability may be misleading.

Keywords

Cognitive abilityTest scoresResponse timesDrift-diffusion modelChoice-process data

JEL classification

C41: Duration Analysis • Optimal Timing Strategies C24: Truncated and Censored Models • Switching Regression Models • Threshold Regression Models C91: Laboratory, Individual Behavior D91: Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making‡ J24: Human Capital • Skills • Occupational Choice • Labor Productivity

Information

Type
Original Paper
Information
Journal of the Economic Science Association , Volume 5 , Issue 1: Special Issue: Choice-Process Data in Experimental Economics , August 2019 , pp. 65 - 75
Copyright
Copyright © Economic Science Association 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s40881-019-00064-2) contains supplementary material, which is available to authorized users.

References

Agarwal, S., Mazumder, B. (2013). Cognitive abilities and household financial decision making. American Economic Journal: Applied Economics, 5(1), 193–207.Google Scholar
Benndorf, V., Rau, H. A., Sölch, C. (2018). Minimizing learning behavior in repeated real-effort tasks. Working Paper 343, Center for European, Governance and Economic Development Research, Georg-August-Universität Göttingen.Google Scholar
Borghans, L., Duckworth, A. L., Heckman, J. J., Ter Weel, B. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43(4), 972–1059. 10.1353/jhr.2008.0017Google Scholar
Cattell, R. B. (1971). Abilities: Their structure, growth, and action, Boston: Houghton Mifflin.Google Scholar
Clithero, J. A. (2018). Improving out-of-sample predictions using response times and a model of the decision process. Journal of Economic Behavior & Organization, 148, 344–375. 10.1016/j.jebo.2018.02.007Google Scholar
Dohmen, T., Falk, A., Huffman, D., Sunde, U. (2010). Are risk aversion and impatience related to cognitive ability? American Economic Review, 100(3), 1238–1260. 10.1257/aer.100.3.1238Google Scholar
Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., Stouthamer-Loeber, M. (2011). Role of test motivation in intelligence testing. Proceedings of the National Academy of Sciences, 108(19), 7716–7720. 10.1073/pnas.1018601108Google ScholarPubMed
Gill, D., Prowse, V. (2016). Cognitive ability, character skills, and learning to play equilibrium: A level-k analysis. Journal of Political Economy, 124(6), 1619–1676. 10.1086/688849Google Scholar
Heckman, J., Pinto, R., Savelyev, P. (2013). Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. American Economic Review, 103(6), 2052–2086. 10.1257/aer.103.6.2052Google ScholarPubMed
Heckman, J. J., Stixrud, J., Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482. 10.1086/504455Google Scholar
Krajbich, I., Lu, D., Camerer, C., Rangel, A. (2012). The attentional drift-diffusion model extends to simple purchasing decisions. Frontiers in psychology, 3, 193 10.3389/fpsyg.2012.00193Google ScholarPubMed
Lee, M.-L. T., Whitmore, G. (2006). Threshold regression for survival analysis: Modeling event times by a stochastic process reaching a boundary. Statistical Science, 21(4), 501–513. 10.1214/088342306000000330Google Scholar
Murnane, R., Willett, J. B., Levy, F. (1995). The growing importance of cognitive skills in wage determination. The Review of Economics and Statistics, 77(2), 251–66. 10.2307/2109863Google Scholar
Ofek, E., Yildiz, M., Haruvy, E. (2007). The impact of prior decisions on subsequent valuations in a costly contemplation model. Management Science, 53(8), 1217–1233. 10.1287/mnsc.1060.0689Google Scholar
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59 10.1037/0033-295X.85.2.59Google Scholar
Ratcliff, R., Van Dongen, H. P. A. (2011). Diffusion model for one-choice reaction-time tasks and the cognitive effects of sleep deprivation. Proceedings of the National Academy of Sciences, 108(27), 11285–11290. 10.1073/pnas.1100483108Google ScholarPubMed
Segal, C. (2012). Working when no one is watching: Motivation, test scores, and economic success. Management Science, 58(8), 1438–1457. 10.1287/mnsc.1110.1509Google Scholar
Spiliopoulos, L., Ortmann, A. (2018). The BCD of response time analysis in experimental economics. Experimental Economics, 21(2), 383–433. 10.1007/s10683-017-9528-1Google ScholarPubMed
Vernon, P. A. (1983). Speed of information processing and general intelligence. Intelligence, 7(1), 53–70. 10.1016/0160-2896(83)90006-5Google Scholar
Webb, R. (2019). The (neural) dynamics of stochastic choice. Management Science, 65(1), 230–255. 10.1287/mnsc.2017.2931Google Scholar
Weiss, L. G., Saklofske, D. H., Coalson, D. L., Raiford, S. E. (2010). WAIS-IV clinical use and interpretation: Scientist-practitioner perspectives, San Diego: Academic Press.Google Scholar
Wilcox, N. T. (1993). Lottery choice: Incentives, complexity and decision time. The Economic Journal, 103(421), 1397–1417. 10.2307/2234473Google Scholar
Woodford, M. (2014). Stochastic choice: An optimizing neuroeconomic model. American Economic Review, 104(5), 495–500. 10.1257/aer.104.5.495Google Scholar
Supplementary material: File

Alekseev supplementary material

Appendices
Download Alekseev supplementary material(File)
File 359.4 KB