执行 XGBClassifier 的增量学习

Question

在参考this link 之后，我能够使用XGBoost 成功实现增量学习。我想构建一个分类器并且需要检查预测概率，即predict_proba() 方法。如果我使用XGBoost，这是不可能的。在实现XGBClassifier.fit() 而不是XGBoost.train() 时，我无法执行增量学习。 XGBClassifier.fit() 的 xgb_model 参数采用 XGBoost，而我想提供 XGBClassifier。

由于我需要使用predict_proba()方法，是否可以对XGBClassifier进行增量学习？

工作代码：

import XGBoost as xgb

train_data = xgb.DMatrix(X, y)
model = xgb.train(
    params = best_params, 
    dtrain = train_data, 
)

new_train_data = xgb.DMatrix(X_new, y_new)
retrained_model = xgb.train(
    params     = best_params, 
    dtrain     = new_train_data, 
    xgb_model  = model
)

以上代码运行完美，但没有retrained_model.predict_proba()选项

非工作代码：

import XGBoost as xgb

xgb_model = xgb.XGBClassifier(**best_params)
xgb_model.fit(X, y)

retrained_model = xgb.XGBClassifier(**best_params)
retrained_model.fit(X_new, y_new, xgb_model = xgb_model)

上述代码不起作用，因为它需要加载 XGBoost 模型或 Booster instance XGBoost 模型。

错误追踪：

[11:27:51] WARNING: ../src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
Traceback (most recent call last):
  File "/project/Data_Training.py", line 530, in train
    retrained_model.fit(X_new, y_new, xgb_model = xgb_model)
  File "/home/user/.local/lib/python3.6/site-packages/xgboost/core.py", line 422, in inner_f
    return f(**kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/xgboost/sklearn.py", line 915, in fit
    callbacks=callbacks)
  File "/home/user/.local/lib/python3.6/site-packages/xgboost/training.py", line 236, in train
    early_stopping_rounds=early_stopping_rounds)
  File "/home/user/.local/lib/python3.6/site-packages/xgboost/training.py", line 60, in _train_internal
    model_file=xgb_model)
  File "/home/user/.local/lib/python3.6/site-packages/xgboost/core.py", line 1044, in __init__
    raise TypeError('Unknown type:', model_file)
TypeError: ('Unknown type:', XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
              colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,
              importance_type='gain', interaction_constraints='',
              learning_rate=1, max_delta_step=0, max_depth=3,
              min_child_weight=1, missing=nan, monotone_constraints='()',
              n_estimators=100, n_jobs=32, num_parallel_tree=1, random_state=0,
              reg_alpha=0, reg_lambda=1, scale_pos_weight=1, subsample=0.7,
              tree_method='exact', validate_parameters=1, verbosity=None))

Answer 1

来自文档：

xgb_model – 存储的 XGBoost 模型或“Booster”实例的文件名[.] XGBoost 模型在训练前加载（允许继续训练）。

所以您应该能够使用xgb_model.get_booster() 检索底层Booster 实例并传递它。

---

此外，您可以从原生 xgboost API 中获得预测概率； Booster.predict 在objective='binary:logistic' 时返回概率。