LARGE SAMPLE VARIANCE OF KAPPA IN THE CASE OF DIFFERENT SETS OF RATERS

Published formulas for the large sample variance of the kappa statistic that are appropriate for the case of different sets of raters for different Ss, when each set of raters is selected at random from a larger pool of available raters, are determined to be incorrect. New formulas are derived and checked by Monte Carlo simulation. Kappa is shown to be identical, except for terms that go to zero as the number of Ss increases, to the intraclass correlation coefficient resulting from applying a one-way ANOVA to the data. (10 ref) (PsycINFO Database Record (c) 2006 APA, all rights reserved). © 1979 American Psychological Association.