How Much Do Rival Diagnostic Criteria Overlap? Recovering the 2×2 Table Behind a Published Kappa
Two diagnostic criteria applied to one population yield a 2×2 table. Studies often print an agreement statistic such as Cohen’s kappa rather than the table, and the same kappa can arise when one criterion’s positive class is nested within the other’s, so that switching criteria only adds diagnoses, and when each criterion identifies patients the other does not. A sample size, both positive counts and kappa determine the table; because printed figures are rounded, they identify a finite set of tables, which we enumerate in exact rational arithmetic.
Applied to sixteen published comparisons across eight conditions, nine were eligible: seven identify a single table, one identifies eleven, and one admits none. Where four comparisons print both their tables and kappa, the enumeration returns exactly those tables. Two readings of the DSM-5 delirium criteria, published at kappa = 0.29, identify one table in which the strict class is nested within the relaxed class, and 0.29 is the highest kappa those prevalences allow. In a study of eleven gestational diabetes criteria, the 55 pairwise tables are nested in 21 pairs and crossing in 34, across almost the same range of kappa.
enum2x2
Enumerates every integer 2×2 table whose statistics round back to a set of printed figures, in exact rational arithmetic. Python and R implementations; the R port is verified against the Python package on 11,296 cases and covers kappa and observed agreement.