具有评分准确性的 RandomizedSearchCV 会导致错误：评分失败。这些参数在此训练测试分区上的分数将设置为 nan答案

【问题标题】：RandomizedSearchCV with scoring accuracy leads to error:Scoring failed.The score on this train-test partition for these parameters will be set to nan具有评分准确性的 RandomizedSearchCV 会导致错误：评分失败。这些参数在此训练测试分区上的分数将设置为 nan
【发布时间】：2022-03-26 08:40:17
【问题描述】：

我想通过使用 RandomizedSearchCV 在准确性方面找到一个好的神经网络实例，因为任务是解决二进制分类问题。不幸的是，我收到了错误消息

Scoring failed. The score on this train-test partition for these parameters will be set to nan.

这是我的实现：

# Define neural network instance
def build_model(n_hidden_layers=2, n_neurons=77, dropout_rate=0.5 ,optimizer='adam', input_shape=77, activation_hidden="relu", activation_output="sigmoid",loss='binary_crossentropy',metrics=['binary_accuracy'],hidden_weight_initializer="he_normal",output_weight_initializer="glorot_normal",l1=0,l2=0,use_batch_norm=0):
    model = keras.models.Sequential()
    model.add(keras.layers.InputLayer(input_shape=input_shape))
    for layer in range(n_hidden_layers):
        model.add(keras.layers.Dense(n_neurons, activation=activation_hidden, kernel_initializer=hidden_weight_initializer, kernel_regularizer=tf.keras.regularizers.l1_l2(l1,l2)))
        model.add(keras.layers.Dropout(dropout_rate))
        if use_batch_norm == 1:
            model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.Dense(1,activation=activation_output, kernel_initializer=output_weight_initializer))
    model.compile(loss=loss, optimizer=optimizer, metrics=metrics)
    return model

# Dreate wrapper class for RandomizedSearchCV
keras_reg = keras.wrappers.scikit_learn.KerasRegressor(build_model)

# Define hyperparameter spaces for trained neural network instances
param_distribs = {
    "n_hidden_layers": [1,2, 3,4,5],
    "n_neurons": [x for x in range(10,100)],
    "dropout_rate": [0, 0.1, 0.2, 0.3, 0.4, 0.5],
    "use_batch_norm": [0,1],            
   # "optimizer": ['adam',],
    "activation_hidden": ['relu','elu','selu'],# 'relu','','elu','selu',,'LeakyRelU(alpha=0.2)','PReLU(alpha_initializer=Constant(value=0.25))'
   # "activation_output": ['relu','sigmoid'],
   # "loss": ['binary_crossentropy']
   # "l1": 
   # "l2": 

}


from sklearn.metrics import make_scorer, precision_score, accuracy_score

precision = make_scorer(precision_score, pos_label="donated")
accuracy = make_scorer(accuracy_score, pos_label="donated")

# Use RandomizedSearchCV to find model instance with best performance on training data
rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=2, cv=2,scoring="accuracy")#, scoring=accuracy,random_state=1)#iter=10,cv=3
rnd_search_cv.fit(X_train, y_train, epochs=10,#100
                  validation_data=(X_test, y_test),
                  callbacks=[keras.callbacks.EarlyStopping(patience=5)],
                  batch_size=256)

【问题讨论】：

标签： python machine-learning neural-network gridsearchcv

【解决方案1】：

我试图做同样的事情并遇到了同样的问题。我发现问题出在RandomizedSearchCV() 中提供的scorer 对象。损失函数应该在编译模型时在你的build_model()函数中指定，并且不能由RandomizedSearchCV()提供。

如果您想使用简单的损失函数或度量标准，如 'accuray'，这应该很容易，因为 keras 已经提供了它。如果你想要一个更具体的损失函数，你可能需要自己构建它，并确保它满足 tensorflow 文档https://www.tensorflow.org/api_docs/python/tf/keras/Model 中描述的要求。

【讨论】：