Hostname: page-component-7b7567c8c-9cwtv Total loading time: 0 Render date: 2026-11-19T07:12:10.840Z Has data issue: false hasContentIssue false

Agreement between Two Independent Groups of Raters

Published online by Cambridge University Press:  01 January 2025

Sophie Vanbelle*
Affiliation:
University of Liege
Adelin Albert
Affiliation:
University of Liege
*
Requests for reprints should be sent to Sophie Vanbelle, Medical Informatics and Biostatistics, University of Liege, CHU Sart Tilman (B23), 4000 Liege, Belgium. E-mail: sophie.vanbelle@ulg.ac.be

Abstract

We propose a coefficient of agreement to assess the degree of concordance between two independent groups of raters classifying items on a nominal scale. This coefficient, defined on a population-based model, extends the classical Cohen’s kappa coefficient for quantifying agreement between two raters. Weighted and intraclass versions of the coefficient are also given and their sampling variance is determined by the Jackknife method. The method is illustrated on medical education data which motivated the research.

Information

Type
Theory and Methods
Copyright
Copyright © 2009 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable