Hostname: page-component-74656f9884-npj9w Total loading time: 0 Render date: 2026-11-10T01:13:10.672Z Has data issue: false hasContentIssue false
  • English
  • Français

Application of Sequential Interval Estimation to Adaptive Mastery Testing

Published online by Cambridge University Press:  01 January 2025

Yuan-chin Ivan Chang
Yuan-chin Ivan Chang*
Affiliation:
Institute of Statistical Science, Academia Sinica, Taipei, Taiwan
*
Requests for reprints should be sent to Yuan-chin Ivan Chang, Institute of Statistical Science, Academia Sinica, Taipei, Taiwan. E-mail: ycchang@sinica.edu.tw
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we apply sequential one-sided confidence interval estimation procedures with β-protection to adaptive mastery testing. The procedures of fixed-width and fixed proportional accuracy confidence interval estimation can be viewed as extensions of one-sided confidence interval procedures. It can be shown that the adaptive mastery testing procedure based on a one-sided confidence interval with β-protection is more efficient in terms of test length than a testing procedure based on a two-sided/fixed-width confidence interval. Some simulation studies applying the one-sided confidence interval procedure and its extensions mentioned above to adaptive mastery testing are conducted. For the purpose of comparison, we also have a numerical study of adaptive mastery testing based on Wald's sequential probability ratio test. The comparison of their performances is based on the correct classification probability, averages of test length, as well as the width of the “indifference regions.” From these empirical results, we found that applying the one-sided confidence interval procedure to adaptive mastery testing is very promising.

Keywords

adaptive mastery testingconfidence intervalβ-protectionfixed-width confidence intervalfixed proportional accuracySPRTindifference region

Information

Type
Theory and Methods
Information
Psychometrika , Volume 70 , Issue 4 , December 2005 , pp. 685 - 713
Copyright
Copyright © 2005 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Anscombe, F.J. (1952). Large-sample theory of sequential estimation. Proceedings of the Cambridge Philosophical Society, 48, 600–607.CrossRefGoogle Scholar
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In Lord, F.M., Novic, M.R. (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
Chang, Y.-C.I. (1999). Strong consistency of maximum quasi-likelihood estimate of generalized linear models via a last time. Statistics and Probability Letters, 45, 237–246.CrossRefGoogle Scholar
Chang, Y.-C.I. (2001). Sequential confidence regions of generalized linear models with adaptive designs. Journal of Statistical Planning and Inference, 93, 277–293.CrossRefGoogle Scholar
Chang, Y.-C.I. (2003). Application of sequential probability ratio test to computerized criterion-referenced Testing. Technical Report 2003-01. Academia Sinica, Taiwan: Institute of Statistical Science.Google Scholar
Chang, Y.-C.I., Martinsek, A. (1992). Fixed size confidence regions for parameters of a logistic regression model. Annals of Statistics, 20, 1953–1969.CrossRefGoogle Scholar
Chang, H.-H., Ying, Z. (1999). a-Stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 263–278.CrossRefGoogle Scholar
Chang, Y.-C.I. & Ying, Z. (2003). Sequential estimation in variable length computerized adaptive testing. Journal of Statistical Planning and Inference, (in Press).Google Scholar
Chow, Y.S. (1965). Local convergence of martingales and the law of large numbers. Annals of Mathematical Statistics, 36, 552–558.CrossRefGoogle Scholar
Chow, Y.S., Robbins, H. (1965). On the asymptotic theory of fixed-width sequential intervals for the mean. Annals of Mathematical Statistics, 36, 457–462.CrossRefGoogle Scholar
Chow, Y.S., Teicher, H. (1988). Probability theory: independence, interchangeability, martingales. New York: Springer-Verlag.CrossRefGoogle Scholar
Epstein, K.I. & Knerr, C.S. (1977). Applications of sequential testing procedures to performance testing. In Weiss, D.J. (Ed.), Proceedings of the 1977 computerized adaptive testing conference (pp. 249–270).Google Scholar
Ferguson, R.L. (1969). The development, implementation, and evaluation of a computer-assisted branched test for a program of individually prescribed instruction. Doctoral dissertation, University of Pittsburgh. Dissertation Abstracts International, 30-09A, 3856. (University Microfilms No. 70–4530).Google Scholar
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.Google Scholar
Ghosh, B.K., Sen, P.K. (1991). Handbook of sequential analysis. New York: Marcel Dekker.Google Scholar
Ghosh, M., Mukhopadhyay, N., Sen, P.K. (1997). Sequential estimation. New York: Wiley.CrossRefGoogle Scholar
Glas, C.A.W. & Vos, H.J. (1998). Adaptive mastery testing using the Rasch model and Bayesian sequential decision theory. Research Report 98-15. Department of Educational Measurement and Data Analysis, University of Twente, Enschede, The Netherlands.Google Scholar
Juhlin, K.D. (1985). Sequential and non-sequential confidence intervals with guaranteed coverage probability and beta-protection. PhD. Dissertation. University of Illinois.Google Scholar
Kingsbury, G.G., Weiss, D.J. (1983). A comparison of IRT-based adaptive mastery testing and sequential mastery testing procedure. In Weiss, D.J. (Eds.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 257–283). New York: Academic Press.Google Scholar
Lord, F.M. (1971). Tailored testing, an application of stochastic approximation. Journal of the American Statistical Association, 66, 707–711.CrossRefGoogle Scholar
Pollard, D. (1984). Convergence of stochastic processes. New York: Springer-Verlag.CrossRefGoogle Scholar
Rasch, G. (1960). Probabilitistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.Google Scholar
Reckase, M.D. (1983). A procedure for decision making using tailored testing. In Weiss, D.J. (Eds.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 237–255). New York: Academic Press.Google Scholar
Siegmund, D. (1985). Sequential analysis. New York: Springer-Verlag.CrossRefGoogle Scholar
Spray, J.A. (1993). Multiple-category classification using a sequential probability ratio test. ACT Research Report Series 93-7. The American College Testing Program, Iowa.Google Scholar
Spray, J.A., Reckase, M.D. (1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405–414.CrossRefGoogle Scholar
Tartakovsky, A. (1998). Asymptotic optimality of certain multihypothesis sequential tests: non-i.i.d. case. Statistical Inference for Stochastic Processes, 1, 265–295.CrossRefGoogle Scholar
Wainer, H. (2000). Computerized adaptive testing: A primer (2nd ed.). Hillsdale, NJ: Erlbaum.CrossRefGoogle Scholar
Wald, A. (1947). Sequential analysis. New York: Wiley.Google Scholar
Weiss, D.J. (1983). New horizons in testing: Latent trait test theory and computerized adaptive testing. New York: Academic Press.Google Scholar
Wijsman, R. (1981). Confidence sets Communications in Statistic—heavy and Methods, Series A, based on sequential tests. 10, 2137–2147.Google Scholar
Wijsman, R. (1982). Sequential confidence sets: Estimation-oriented versus test-oriented construction. In Gupta, S.S. & Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics (Vol. III, 2nd ed. pp. 435–450), New York: Academic Press.CrossRefGoogle Scholar
Wijsman, R. (1986). Sequential confidence intervals with β-protection in one-parameter families. In Van Ryzin, J. (Ed.) Adaptive statistical procedures and related topics. Lecture Notes—Monograph Series Vol. 8, Hayward, CA: Insitute of Mathematical Statistics.Google Scholar
Woodroofe, M. (1982). Nonlinear renewal theory in sequential analysis. CBMS-NSF Regional Conference Series in Applied Mathematics. Philadephia: SIAM.CrossRefGoogle Scholar