Hostname: page-component-6898cfcb55-w5hgh Total loading time: 0 Render date: 2026-11-11T17:12:29.121Z Has data issue: false hasContentIssue false
  • English
  • Français

Cultural Consensus Theory for the Ordinal Data Case

Published online by Cambridge University Press:  01 January 2025

Royce Anders and
William H. Batchelder
Royce Anders*
Affiliation:
University of California, Irvine
William H. Batchelder
Affiliation:
University of California, Irvine
*
Requests for reprints should be sent to Royce Anders, University of California, Irvine, USA. E-mail: andersr@uci.edu
Get accessRights & Permissions [Opens in a new window]

Abstract

A Cultural Consensus Theory approach for ordinal data is developed, leading to a new model for ordered polytomous data. The model introduces a novel way of measuring response biases and also measures consensus item values, a consensus response scale, item difficulty, and informant knowledge. The model is extended as a finite mixture model to fit both simulated and real multicultural data, in which subgroups of informants have different sets of consensus item values. The extension is thus a form of model-based clustering for ordinal data. The hierarchical Bayesian framework is utilized for inference, and two posterior predictive checks are developed to verify the central assumptions of the model.

Keywords

cultural consensus theorylatent class modelslatent trait modelsrater modelsordinal datasignal detection theorymixture modelingclustering

Information

Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 1 , March 2015, pp. 151 - 181
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Anders, R. (2013). CCTpack: cultural consensus theory applications to data. R package version 0.9.Google Scholar
Anders, R., & Batchelder, W.H. (2012). Cultural consensus theory for multiple consensus truths. Journal for Mathematical Psychology, 56, 452–469.CrossRefGoogle Scholar
Batchelder, W.H., & Anders, R. (2012). Cultural consensus theory: comparing different concepts of cultural truth. Journal of Mathematical Psychology, 56, 316–332.CrossRefGoogle Scholar
Batchelder, W.H., & Romney, A.K. (1986). The statistical analysis of a general condorcet model for dichotomous choice situations. In Grofman, B., & Owen, G. Information pooling and group decision making: proceedings of the second University of California Irvine conference on political economy (pp. 103–112). Greenwich: JAI Press.Google Scholar
Batchelder, W.H., & Romney, A.K. (1988). Test theory without an answer key. Psychometrika, 53, 71–92.CrossRefGoogle Scholar
Batchelder, W.H., & Romney, A.K. (1989). New results in test theory without an answer key. In Roskam, (Ed.), Mathematical psychology in progress (pp. 229–248). Heidelberg: Springer.CrossRefGoogle Scholar
Buhrmester, M., Kwang, T., & Gosling, S.D. (2011). Amazon’s mechanical turk a new source of inexpensive, yet high-quality, data?. Perspectives on Psychological Science, 6, 3–5.CrossRefGoogle Scholar
Comrey, A.L. (1962). The minimum residual method of factor analysis. Psychological Reports, 11, 15–18.CrossRefGoogle Scholar
De Boeck, P. (2008). Random item IRT models. Psychometrika, 73, 533–559.CrossRefGoogle Scholar
DeCarlo, L.T. (2005). A model of rater behavior in essay grading based on signal detection theory. Journal of Educational Measurement, 42, 53–76.CrossRefGoogle Scholar
Fischer, G.H., & Molenaar, I.W. (1995). Rasch models: recent developments and applications. New York: Springer.CrossRefGoogle Scholar
Fox, C.R., & Tversky, A. (1995). Ambiguity aversion and comparative ignorance. The Quarterly Journal of Economics, 110, 585–603.CrossRefGoogle Scholar
Fox, J. (2013). Polycor: polychoric and polyserial correlations. R package version 0.7-8.Google Scholar
Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (2004). Bayesian data analysis (2nd ed.). Boca Raton: Chapman and Hall/CRC.Google Scholar
Gonzalez, R., & Wu, G. (1999). On the shape of the probability weighting function. Cognitive Psychology, 38, 129–166.CrossRefGoogle ScholarPubMed
Green, D.M., & Swets, J.A. (1966). Signal detection theory and psychophysics. New York: Wiley.Google Scholar
Hruschka, D.J., Kalim, N., Edmonds, J., & Sibley, L. (2008). When there is more than one answer key: cultural theories of postpartum hemorrhage in Matlab, Bangladesh. Field Methods, 20, 315–337.CrossRefGoogle Scholar
Johnson, V.E., & Albert, J.H. (1999). Ordinal data modeling. Statistics for social science and public policy. Berlin: Springer.Google Scholar
Karabatsos, G., & Batchelder, W.H. (2003). Markov chain estimation methods for test theory without an answer key. Psychometrika, 68, 373–389.CrossRefGoogle Scholar
Kruschke, J.K. (2011). Doing Bayesian data analysis: a tutorial with R and BUGS. Amsterdam: Elsevier/Academic Press.Google Scholar
Lancaster, H., & Hamdan, M. (1964). Estimation of the correlation coefficient in contingency tables with possibly nonmetrical characters. Psychometrika, 29, 383–391.CrossRefGoogle Scholar
Lee, M.D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 1–7.CrossRefGoogle Scholar
Lord, F.M., Novick, M.R., & Birnbaum, A. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.Google Scholar
Macmillan, N.A., & Creelman, C.D. (2005). Detection theory: a users guide (2nd ed.). Mahwah: Erlbaum.Google Scholar
Nering, M.L., & Ostini, R. (2011). Handbook of polytomous item response theory models. New York: Taylor and Francis.CrossRefGoogle Scholar
Patz, R.J., Junker, B.W., Johnson, M.S., & Mariano, L.T. (2002). The hierarchical rater model for rated test items and its application to large-scale educational assessment data. Journal of Educational and Behavioral Statistics, 27, 341–384.CrossRefGoogle Scholar
Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling.Google Scholar
Plummer, M. (2012). Rjags: Bayesian graphic models using MCMC. R package version 3.2.0. http://CRAN.R-project.org/package=rjags.Google Scholar
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Denmarks Paedagogiske Institute.Google Scholar
Revelle, W. (2012). psych: procedures for psychological, psychometric, and personality research. Northwestern University Evanston, Illinois. R package version 1.2.1.Google Scholar
Rigdon, E.E. (2010). Polychoric correlation coefficient. In Salkind, N.J. Encyclopedia of research design (pp. 1046–1049). Thonsand Oaks: Sage.Google Scholar
Romney, A.K., & Batchelder, W.H. (1999). Cultural consensus theory. In Wilson, R., & Keil, F. The MIT encyclopedia of the cognitive sciences (pp. 208–209). Cambridge: MIT Press.Google Scholar
Romney, A.K., Batchelder, W.H., & Weller, S.C. (1987). Recent applications of cultural consensus theory. American Behavioral Scientist, 31, 163–177.CrossRefGoogle Scholar
Romney, A.K., Weller, S.C., & Batchelder, W.H. (1986). Culture as consensus: a theory of culture and informant accuracy. American Anthropologist, 88, 313–338.CrossRefGoogle Scholar
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement.CrossRefGoogle Scholar
Spearman, C.E. (1904). ‘General intelligence’ objectively determined and measured. The American Journal of Psychology, 15, 72–101.CrossRefGoogle Scholar
Spiegelhalter, D.J., Best, N.G., Carlin, B.P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society, Series B, 6, 583–640.CrossRefGoogle Scholar
Sprouse, J., Wagers, M., & Phillips, C. (2012). A test of the relation between working-memory capacity and syntactic island effects. Language, 88, 82–123.CrossRefGoogle Scholar
Stephens, M. (2000). Dealing with label switching in mixture models. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 62, 795–809.CrossRefGoogle Scholar
Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408.CrossRefGoogle Scholar
van der Linden, W.J., & Hambleton, R.K. (1997). Handbook of modern item response theory. Berlin: Springer.CrossRefGoogle Scholar
Weller, S.W. (2007). Cultural consensus theory: applications and frequently asked questions. Field Methods, 19, 339–368.CrossRefGoogle Scholar
Zhang, H., & Maloney, L.T. (2012). Ubiquitous log odds: a common representation of probability and frequency distortion in perception, action and cognition. Frontiers in Neuroscience, 6.CrossRefGoogle Scholar