使用miceforest 进行插补并在RandomState.choice() 上出错

Question

我正在处理一个使用不同插补技术的 Kaggle 数据集，我从miceforest 包中得到一个我不理解的错误。

完整的 Colab here，但要点是我使用的数据是来自我的 X_train 数据集的数字特征。

X = df_agg.drop(['TARGET','SK_ID_CURR'], axis = 1)
y = df_agg.TARGET

X_train_raw, X_test_raw, y_train, y_test = train_test_split(
  X, y, test_size=0.10, random_state=42, stratify=y)


X_train_raw, X_dev_raw, y_train, y_dev = train_test_split(
  X_train_raw, y_train,
  test_size=1/9.,
  random_state=42,
  stratify=y_train
)
num_features = X_train_raw.select_dtypes(include=['int64', 'float64']).columns 



kernel = mf.MultipleImputedKernel(
  data=X_train_raw[num_features],
  save_all_iterations=True,
  random_state=1991
)

这是错误：

ValueError                                Traceback (most recent call last)
<ipython-input-20-b6bd23e78d87> in <module>
      2   data=X_train_raw[num_features],
      3   save_all_iterations=True,
----> 4   random_state=1991
      5 )

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/miceforest/MultipleImputedKernel.py in __init__(self, data, datasets, variable_schema, mean_match_candidates, save_all_iterations, save_models, random_state)
     56                 save_all_iterations=save_all_iterations,
     57                 save_models=save_models,
---> 58                 random_state=random_state,
     59             )
     60         )

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/miceforest/KernelDataSet.py in __init__(self, data, variable_schema, mean_match_candidates, save_all_iterations, save_models, random_state)
     88             mean_match_candidates=mean_match_candidates,
     89             save_all_iterations=save_all_iterations,
---> 90             random_state=random_state,
     91         )
     92 

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/miceforest/ImputedDataSet.py in __init__(self, data, variable_schema, mean_match_candidates, save_all_iterations, random_state)
     87             self.imputation_values[var] = {
     88                 0: self._random_state.choice(
---> 89                     data[var].dropna(), size=self.na_counts[var]
     90                 )
     91             }

mtrand.pyx in numpy.random.mtrand.RandomState.choice()

ValueError: 'a' cannot be empty unless no samples are taken

Answer 1

我是这个包的维护者——你能把你运行这个包的 kaggle 数据链接给我吗？作为快速检查 - 当您将数据传递给此函数时，您能否确保数据中没有 100% 缺失值的列？将空对象传递给 numpy.random.choice() 时会出现此错误，因此 data[var].dropna() 必须为空。