Hostname: page-component-6898cfcb55-cbk24 Total loading time: 0 Render date: 2026-11-11T05:23:48.600Z Has data issue: false hasContentIssue false
  • English
  • Français

Uniform Test Assembly

Published online by Cambridge University Press:  01 January 2025

Dmitry I. Belov
Dmitry I. Belov*
Affiliation:
Law School Admission Council
*
Requests for reprints should be sent to Dmitry I. Belov, Psychometric Research, Law School Admission Council, 662 Penn Street, Newtown, PA 18940, USA. E-mail: dbelov@LSAC.org; belovd@mail.ru
Get accessRights & Permissions [Opens in a new window]

Abstract

In educational practice, a test assembly problem is formulated as a system of inequalities induced by test specifications. Each solution to the system is a test, represented by a 0–1 vector, where each element corresponds to an item included (1) or not included (0) into the test. Therefore, the size of a 0–1 vector equals the number of items n in a given item pool. All solutions form a feasible set—a subset of 2n vertices of the unit cube in an n-dimensional vector space. Test assembly is uniform if each test from the feasible set has an equal probability of being assembled. This paper demonstrates several important applications of uniform test assembly for educational practice. Based on Slepian’s inequality, a binary program was analytically studied as a candidate for uniform test assembly. The results of this study establish a connection between combinatorial optimization and probability inequalities. They identify combinatorial properties of the feasible set that control the uniformity of the binary programming test assembly. Computer experiments illustrating the concepts of this paper are presented.

Keywords

combinatorial optimizationbinary programmingprobability inequalitiesSlepian’s inequalitytest assemblyitem pool analysis

Information

Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 1 , March 2008, pp. 21 - 38
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Armstrong, R.D., Jones, D.H., & Kunce, C.S. (1998). IRT test assembly using network-flow programming. Applied Psychological Measurement, 22, 237–247.10.1177/01466216980223004CrossRefGoogle Scholar
Belov, D.I. (2005). Inverse problem of item pool usability in computerized adaptive testing. Presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada, April.Google Scholar
Belov, D.I., & Armstrong, R.D. (2005). Monte Carlo test assembly for item pool analysis and extension. Applied Psychological Measurement, 29, 239–261.10.1177/0146621605275413CrossRefGoogle Scholar
Belov, D.I., & Armstrong, R.D. (2005b). A Monte Carlo approach for evaluating and designing multi-stage adaptive tests. Presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada, April.Google Scholar
Belov, D.I., & Armstrong, R.D. (2006). A constraint programming approach to extract the maximum number of non-overlapping test forms. Computational Optimization and Applications, 33 2/3319–332.10.1007/s10589-005-3058-zCrossRefGoogle Scholar
Belov, D.I., & Armstrong, R.D. (in press). A Monte Carlo approach to the design, assembly and evaluation of multi-stage adaptive tests. Applied Psychological Measurement.Google Scholar
Boekkooi-Timminga, E. (1990). The construction of parallel tests from IRT-based item banks. Journal of Educational Statistics, 15, 129–145.10.3102/10769986015002129CrossRefGoogle Scholar
Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness, New York: Freeman.Google Scholar
ILOG, Inc. (2003). CPLEX 9.0 [Computer program and manual], Mountain View: IL OS, Inc..Google Scholar
Lord, F.M. (1980). Applications of item response theory to practical testing problems, Hillsdale: Lawrence Erlbaum.Google Scholar
Luecht, R.M. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224–236.10.1177/01466216980223003CrossRefGoogle Scholar
Luecht, R.M. & Hirsch, T.M. (1992). Item selection using an average growth approximation of target information functions. Applied Psychological Measurement, 16, 41–51.10.1177/014662169201600104CrossRefGoogle Scholar
Slepian, D. (1962). The one-sided barrier problem for Gaussian noise. Bell System Technical Journal, 41, 463–501.10.1002/j.1538-7305.1962.tb02419.xCrossRefGoogle Scholar
Theunissen, T.J.J.M. (1985). Binary programming and test design. Psychometrika, 50, 411–420.10.1007/BF02296260CrossRefGoogle Scholar
Tong, Y.L. (1980). Probability inequalities in multivariate distributions, New York: Academic Press.Google Scholar
Tong, Y.L. (1990). The multivariate normal distribution, New York: Springer.10.1007/978-1-4613-9655-0CrossRefGoogle Scholar
van der Linden, W.J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195–211.CrossRefGoogle Scholar
van der Linden, W.J. (2005). Linear models for optimal test design, New York: Springer.10.1007/0-387-29054-0CrossRefGoogle Scholar
van der Linden, W.J. (2005b). Personal communication.Google Scholar
van der Linden, W.J., & Adema, J.J. (1998). Simultaneous assembly of multiple test forms. Journal of Educational Measurement, 35, 185–198.10.1111/j.1745-3984.1998.tb00533.xCrossRefGoogle Scholar
van der Linden, W.J., Ariel, A., & Veldkamp, B.P. (2006). Assembling a CAT item pool as a set of linear tests. Journal of Educational and Behavioral Statistics, 31(1), 81–99.10.3102/10769986031001081CrossRefGoogle Scholar