The measurement of observer agreement for categorical data

J. R. Landis, Gary G Koch

Research output: Contribution to journalArticle

Abstract

This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.

LanguageEnglish (US)
Pages159-174
Number of pages16
JournalBiometrics
Volume33
Issue number1
DOIs
StatePublished - Jan 1 1977

Fingerprint

Nominal or categorical data
Observer
Marginal Homogeneity
statistics
Statistics
Multivariate Data
multivariate analysis
Test Statistic
Proportion
Multivariate Analysis
testing
First-order
Methodology
methodology

ASJC Scopus subject areas

  • Statistics and Probability
  • Biochemistry, Genetics and Molecular Biology(all)
  • Immunology and Microbiology(all)
  • Agricultural and Biological Sciences(all)
  • Applied Mathematics

Cite this

The measurement of observer agreement for categorical data. / Landis, J. R.; Koch, Gary G.

In: Biometrics, Vol. 33, No. 1, 01.01.1977, p. 159-174.

Research output: Contribution to journalArticle

Landis, J. R. ; Koch, Gary G. / The measurement of observer agreement for categorical data. In: Biometrics. 1977 ; Vol. 33, No. 1. pp. 159-174.
@article{df4ff77e6299426384fbe5cdcbe614ad,
title = "The measurement of observer agreement for categorical data",
abstract = "This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.",
author = "Landis, {J. R.} and Koch, {Gary G}",
year = "1977",
month = "1",
day = "1",
doi = "10.2307/2529310",
language = "English (US)",
volume = "33",
pages = "159--174",
journal = "Biometrics",
issn = "0006-341X",
publisher = "Wiley-Blackwell",
number = "1",

}

TY - JOUR

T1 - The measurement of observer agreement for categorical data

AU - Landis, J. R.

AU - Koch, Gary G

PY - 1977/1/1

Y1 - 1977/1/1

N2 - This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.

AB - This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.

UR - http://www.scopus.com/inward/record.url?scp=0017360990&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0017360990&partnerID=8YFLogxK

U2 - 10.2307/2529310

DO - 10.2307/2529310

M3 - Article

VL - 33

SP - 159

EP - 174

JO - Biometrics

T2 - Biometrics

JF - Biometrics

SN - 0006-341X

IS - 1

ER -