Hostname: page-component-7b7567c8c-2n9c9 Total loading time: 0 Render date: 2026-11-16T08:05:18.344Z Has data issue: false hasContentIssue false

Standard Error of Ability Estimates and the Classification Accuracy and Consistency of Binary Decisions

Published online by Cambridge University Press:  01 January 2025

Ying Cheng*
Affiliation:
University of Notre Dame
Cheng Liu
Affiliation:
University of Notre Dame
John Behrens
Affiliation:
Pearson
*
Correspondence should be sent to Ying Cheng, Department of Psychology, University of Notre Dame, 118 Haggar Hall, Notre Dame, IN 46556, USA. E-mail: ycheng4@nd.edu

Abstract

While estimation bias is a primary concern in psychological and educational measurement, the standard error is of equal importance in linking key aspects of the assessment structure, especially when the assessment goal concerns the classification of individuals into categories (e.g., master/non-mastery). In this paper, we show analytically how standard error of ability estimates affects expected classification accuracy and consistency when the decision is binary. When standard error decreases, the conditional classification accuracy and consistency increase. Given an examinee population and a cut score, smaller standard error over the entire latent trait continuum guarantees higher overall expected classification accuracy and consistency. We were also able to show the interrelationship between standard error, the expected classification consistency, and reliability. Utilizing the relationship between standard error and expected classification accuracy and consistency, we derive the upper bounds of the overall expected classification accuracy and consistency of a fixed-length computerized adaptive test. The lower bound of the expected classification accuracy and consistency is also derived given a number of stopping rules of variable-length computerized adaptive testing. Implications of these analytical results on operational tests are discussed.

Information

Type
Original Paper
Copyright
Copyright © 2014 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable