H2O R 如何处理预测时未训练的级别？

Question

我有一组带有结果列的 CSV 要训练，还有一组没有结果列的测试 CSV。

library(h2o)
h2o.init()

train <- read.csv(train_file, header=T)
train.h2o <- as.h2o(train)
y <- "Result"
x <- setdiff(names(train.h2o), y)

model <- h2o.deeplearning(x = x,
                          y = y,
                          training_frame = train.h2o,
                          model_id = "my_model",
                          epochs = 5000,
                          hidden = c(50),
                          stopping_rounds=5,
                          stopping_metric="misclassification", 
                          stopping_tolerance=0.001,
                          seed = 1)



test <- read.csv(test_file, header=T)
test.h2o <- as.h2o(test)

pred <- h2o.predict(model,test.h2o)

当我尝试使用测试数据预测结果时，我得到了一堆错误，例如：

1: In doTryCatch(return(expr), name, parentenv, handler) :
Test/Validation dataset column 'ColumnName' has levels not trained on: [ABCD, BCDE]

H2O 过去能够处理测试中存在的数据，但不能处理训练期间的数据。我在网上找到了一些他们说的帖子。但这对我不起作用。

如何避免这些错误，并预测测试数据的值？

Answer 1

有两种方法你可以试试：

使用factor 与character 相反

在将数据输入机器学习函数之前，您可以结合训练和测试数据，并将character 变量转换为factor。

因此，即使您稍后拆分组合数据，唯一值也会被记录为级别信息。

library(h2o)

h2o.init()

#using dummy data as combined training and testing data
prostatePath = system.file("extdata", "prostate.csv", package = "h2o")
prostate.hex = h2o.importFile(path = prostatePath, destination_frame = "prostate.hex")

#assuming GLEASON is the character variable, and transform it to factor
prostate.hex$GLEASON <- h2o.asfactor(prostate.hex$GLEASON)

#split data such that 0,4,5,8 only in test set, and not in train set.
h2o.test <- prostate.hex[prostate.hex$GLEASON %in% c("0","4","5","8"),]
h2o.train <- prostate.hex[!prostate.hex$GLEASON %in% c("0","4","5","8"),]

#train model
model <- h2o.glm(y = "CAPSULE", x = c("AGE","RACE","PSA","DCAPS","GLEASON"), training_frame = h2o.train,
       family = "binomial", nfolds = 0)

#predict without error
pred <- predict(model,h2o.test)

明确使用one-hot-encoding

我知道h2o 机器学习函数提供了内部编码方法（通过categorical_encoding 参数），包括one-hot-encoding，它将字符变量转换为大量1/0 整数变量。

与隐式使用此技术相反，您可以显式使用它。因此，训练中不存在的那些级别将不会在模型中使用。测试中的新级别根本不用于预测。