用 psych 计算 Cohen 的 kappa 时发出警告

Question

在我的db 数据框中，我通过scoreA 和scoreB 两种方式计算分数。然后我得到它们的五分位数，我想得到它们之间的 Cohen 的 kappa 系数。

我在 CRAN 中找到了几个库，psych 似乎是最常用的一个，但我对其他可能性持开放态度。

这里有一些代码要测试：

set.seed(1)
df=data.frame(
  scoreA = rnorm(n = 200, mean = 1, sd = .75)*10,
  scoreB = rnorm(n = 200, mean = 5, sd = 3)*10
)
df$quintA = cut(df$scoreA, breaks=quantile(df$scoreA, probs=seq(0,1, by=0.2), na.rm = TRUE), labels=c("Q1", "Q2", "Q3", "Q4", "Q5"), include.lowest = T)
df$quintB = cut(df$scoreB, breaks=quantile(df$scoreB, probs=seq(0,1, by=0.2), na.rm = TRUE), labels=c("Q1", "Q2", "Q3", "Q4", "Q5"), include.lowest = T)
plot(df$quintA, df$quintB, xlab="A", ylab="B")
psych::cohen.kappa(table(df$quintA, df$quintB))

Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha, levels = levels)

Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries 
                  lower estimate upper
unweighted kappa -0.058    0.012 0.083 #My real world estimates are about 0.5
weighted kappa   -0.215   -0.070 0.075

 Number of subjects = 200 
Warning message:
In any(abs(bounds)) : coercing argument of type 'double' to logical

我应该担心警告吗？什么意思？

编辑：我刚刚尝试使用 fmsb 并获得与未加权 kappa 完全相同的值（即使是小数）。如果增重 kappa 出现问题，问题仍然存在。

library(fmsb)
Kappa.test(db$scoreA, db$scoreB)
$Result

    Estimate Cohen kappa statistics and test the null hypothesis that the
    extent of agreement is same as random (kappa=0)

data:  db$scoreA and db$scoreB
Z = 383.67, p-value < 2.2e-16
95 percent confidence interval:
 0.9077865 0.9131626
sample estimates:
[1] 0.9104745

$Judgement
[1] "Almost perfect agreement"

Answer 1

您不必担心自己的结果。您收到警告是因为您的数据是数字的，psych::cohen.kappa 需要逻辑，因此它会尝试将其从原始数据类型强制转换为所需的数据类型。

考虑以下问题。我们正在使用来自两个尺度的模拟数据创建一个数据框。刻度中可能的值是 0 和 1。然后我们运行 cohen.kappa 和 Kappa.test。

set.seed(1984)
db <- data.frame(
  scoreA = rbinom(n = 20, size = 1, prob = .75),
  scoreB = rbinom(n = 20, size = 1, prob = .75)
)

# psych
cohen.kappa(table(db$scoreA, db$scoreB))

Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha, levels = levels)

Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries 
                 lower estimate  upper
unweighted kappa -0.37    -0.21 -0.048
weighted kappa   -0.37    -0.21 -0.048

 Number of subjects = 20 
Warning message:
In any(abs(bounds)) : coercing argument of type 'double' to logical


# fmsb
Kappa.test(db$scoreA, db$scoreB)
$Result

    Estimate Cohen's kappa statistics and test the null hypothesis that the extent of
    agreement is same as random (kappa=0)

data:  db$scoreA and db$scoreB
Z = -0.59134, p-value = 0.7229
95 percent confidence interval:
 -0.9277148  0.5139217
sample estimates:
[1] -0.2068966


$Judgement
[1] "No agreement"

结果几乎相同。这是因为强制是成功的。它确实是这样发生的：

# Original    
db$scoreA
     [1] 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1
# Coerced
    as.logical(db$scoreA)
     [1] FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
    [16]  TRUE FALSE  TRUE  TRUE  TRUE

警告只是告诉你发生了什么，只要你的编码在尺度之间是一致的，就不会有任何问题。