Hostname: page-component-7b7567c8c-fvjsw Total loading time: 0 Render date: 2026-11-17T17:46:29.748Z Has data issue: false hasContentIssue false

Exploring the Effects of Item-Specific Factors in Sequential and IRTree Models

Published online by Cambridge University Press:  01 January 2025

Weicong Lyu*
Affiliation:
University of Wisconsin-Madison
Daniel M. Bolt
Affiliation:
University of Wisconsin-Madison
Samuel Westby
Affiliation:
Northeastern University
*
Correspondence should be made to Weicong Lyu, University of Wisconsin-Madison, 880 Educational Sciences, 1025 West Johnson Street, Madison, WI 53706, USA. Email: wlyu4@wisc.edu

Abstract

Test items for which the item score reflects a sequential or IRTree modeling outcome are considered. For such items, we argue that item-specific factors, although not empirically measurable, will often be present across stages of the same item. In this paper, we present a conceptual model that incorporates such factors. We use the model to demonstrate how the varying conditional distributions of item-specific factors across stages become absorbed into the stage-specific item discrimination and difficulty parameters, creating ambiguity in the interpretations of item and person parameters beyond the first stage. We discuss implications in relation to various applications considered in the literature, including methodological studies of (1) repeated attempt items; (2) answer change/review, (3) on-demand item hints; (4) item skipping behavior; and (5) Likert scale items. Our own empirical applications, as well as several examples published in the literature, show patterns of violations of item parameter invariance across stages that are highly suggestive of item-specific factors. For applications using sequential or IRTree models as analytical models, or for which the resulting item score might be viewed as outcomes of such a process, we recommend (1) regular inspection of data or analytic results for empirical evidence (or theoretical expectations) of item-specific factors; and (2) sensitivity analyses to evaluate the implications of item-specific factors for the intended inferences or applications.

Information

Type
Theory & Methods
Copyright
Copyright © 2023 The Author(s) under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable