H2O GAM 火车：参数“fold_column”不起作用

Question

我无法使用 Python 使参数“fold_column”与 H2OGeneralizedAdditiveEstimator 一起工作。

我需要在 H2O 之外创建折叠，并将完成的 Pandas DataFrame 读入 H2OFrame。在 H2OFrame 中有一个列“fold_number”。我可以遍历折叠并为每个折叠训练模型。但是当使用 fold_column="fold_number" 运行 GAM 训练时，它会失败，“没有足够的数据来创建 2 个随机交叉验证拆分”。但我只是做了这两个模型！即使我通过添加原始的修改副本来增强数据集，它也会失败。使用 H2OGeneralizedLinearEstimator 一切正常。

关于这个的任何提示 - 或者这个错误？

我正在运行 python=3.6.13、h2o=3.32.1.3、pandas=0.25.3、numpy=1.19.5、sklearn=0.24.2。 Java 版本：openjdk 版本“14.0.2”。

import numpy as np
import pandas as pd
from sklearn.datasets import load_boston
import h2o
from h2o.estimators.gam import H2OGeneralizedAdditiveEstimator

h2o.no_progress()
h2o.init()

np.random.seed(42)
boston = load_boston()
y = pd.Series(boston["target"], name="y")
X = pd.DataFrame(boston["data"], columns=boston["feature_names"])  # shape: (506, 13)
myweight = pd.Series(np.random.random_sample((len(y),)), name="myweight2")

predictors = ['CRIM', 'AGE']
gam_columns = ['CRIM']

params = {
    "family": "gaussian",
    "gam_columns": gam_columns,
    'bs': len(gam_columns) * [0],
}

fold = pd.Series(np.append(np.zeros(253), np.ones(253)), dtype=int, index=y.index, name="fold_number")
df0 = pd.concat([y, X, myweight, fold], axis=1)
df = h2o.H2OFrame(python_obj=df0)

# df["fold_number"] = df["fold_number"].asfactor()

for i in [0, 1]:
    mask = df["fold_number"] == i
    df_train = df[~mask, :]
    df_val = df[mask, :]

    model = H2OGeneralizedAdditiveEstimator(**params)
    model.train(
        x=predictors,
        y="y",
        weights_column="myweight2",
        training_frame=df_train,
    )

    print("Finished training for fold_number=", i, ", with validation-RMSE=", model.rmse(df_val))

print("\nStarting training with API option fold_column=")
model2 = H2OGeneralizedAdditiveEstimator(**params)
model2.train(
    x=predictors,
    y="y",
    weights_column="myweight2",
    training_frame=df,
    fold_column="fold_number"
)
print("Finished training with API option fold_column=")

我得到的输出是：

Checking whether there is an H2O instance running at http://localhost:54321 . connected.
--------------------------  ------------------------------------------------

Finished training for fold_number= 0 , with validation-RMSE= 7.33788975630292
Finished training for fold_number= 1 , with validation-RMSE= 7.912477133985602

Starting training with API option fold_column=
Traceback (most recent call last):
  File "/Users/g009655/tmp7/h2otest/test_gam_cv.py", line 55, in <module>
    fold_column="fold_number"
  File "/Users/g009655/Library/Caches/pypoetry/virtualenvs/h2otest-S7Xak4Mg-py3.6/lib/python3.6/site-packages/h2o/estimators/estimator_base.py", line 115, in train
    self._train(parms, verbose=verbose)
  File "/Users/g009655/Library/Caches/pypoetry/virtualenvs/h2otest-S7Xak4Mg-py3.6/lib/python3.6/site-packages/h2o/estimators/estimator_base.py", line 207, in _train
    job.poll(poll_updates=self._print_model_scoring_history if verbose else None)
  File "/Users/g009655/Library/Caches/pypoetry/virtualenvs/h2otest-S7Xak4Mg-py3.6/lib/python3.6/site-packages/h2o/job.py", line 80, in poll
    "\n{}".format(self.job_key, self.exception, self.job["stacktrace"]))
OSError: Job with key $03017f00000132d4ffffffff$_af1219b23ff0642a316d9f092b214dc6 failed with an exception: water.exceptions.H2OIllegalArgumentException: 

Not enough data to create 2 random cross-validation splits. 
Either reduce nfolds, specify a larger dataset (or specify another random number seed, 
if applicable).

stacktrace: 
water.exceptions.H2OIllegalArgumentException: 

Not enough data to create 2 random cross-validation splits. 
Either reduce nfolds, specify a larger dataset (or specify another random number seed, 
if applicable).

    at hex.ModelBuilder.cv_makeWeights(ModelBuilder.java:726)
    at hex.ModelBuilder.computeCrossValidation(ModelBuilder.java:604)
    at hex.glm.GLM.computeCrossValidation(GLM.java:136)
    at hex.ModelBuilder$1.compute2(ModelBuilder.java:379)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1637)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

Closing connection _sid_b4d8 at exit
H2O session _sid_b4d8 closed.

Process finished with exit code 1

Answer 1

我能够重现该错误，但它确实不起作用（nfolds 和 fold_column 似乎都不起作用）。我们会尽快解决这个问题。这是 Jira 票：https://h2oai.atlassian.net/browse/PUBDEV-8163