sklearn异常错误：来自张量流中GridSearchCV的fitfailed警告

Question

我想针对图像分类任务（多类）优化 CNN 模型的超参数。为此，我使用了来自sklearn 的gridSearchCV，但我总是有一堆警告和值错误，如下所示：

/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py", line 531, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "/usr/local/lib/python3.6/dist-packages/keras/wrappers/scikit_learn.py", line 209, in fit
    return super(KerasClassifier, self).fit(x, y, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/keras/wrappers/scikit_learn.py", line 151, in fit
    history = self.model.fit(x, y, **fit_args)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 235, in fit
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 593, in _process_training_inputs
    use_multiprocessing=use_multiprocessing)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_v2.py", line 646, in _process_inputs
    x, y, sample_weight=sample_weights)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2383, in _standardize_user_data
    batch_size=batch_size)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2489, in _standardize_tensors
    y, self._feed_loss_fns, feed_output_shapes)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_utils.py", line 810, in check_loss_and_target_compatibility
    ' while using as loss `' + loss_name + '`. '
**ValueError:** A target array with shape (500, 99) was passed for an output of shape (None, 100) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.

  FitFailedWarning)

对我来说，这个错误可能来自 sklearn，我不确定如何摆脱它以获得最佳参数结果。有没有办法解决这些问题？有什么想法吗？

小例子：

这只是最小的例子：

X = np.random.randn(1000, 2048)
y = np.array([i for i in range(100)]*10) # <- 1D array with target labels

def myCNN():
  model = keras.models.Sequential()
  model.add(keras.layers.Dense(100, input_dim=2048, activation='softmax'))
  model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  return model
  

model = KerasClassifier(build_fn=myCNN)
parameters = { 'epochs': [10, 20, 30],
               'batch_size':[1, 2, 3, 4, 5, 6, 7,8] }              

grid_search = GridSearchCV(estimator=model,
                           param_grid=parameters,
                           scoring='accuracy',
                           cv=2)

grid_search = grid_search.fit(X, y)
print (grid_search.best_params_)

从上面粘贴的错误消息中，我可以假设错误可能来自 sklearn。我有什么办法解决这个问题吗？有什么想法吗？

Answer 1

我建议您可以使用bayes_opt 的贝叶斯优化，它比GridSearchCV 效率更高并且执行速度更快。以下是如何使用贝叶斯优化的快速示例：

import tensorflow as tf
from bayes_opt import BayesianOptimization

def myCNN(fully_conn_size, len_layer_conv, kernel_size, learning_rate, pooling_size, multiply,dropout_rate, beta, activation, batch_normalization, batch_size = 10):
  model = keras.models.Sequential()
  model.add(keras.layers.Dense(100, input_dim=2048, activation='softmax'))
  model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  return model


accbest = 0.0
NN_BAYESIAN = BayesianOptimization(myCNN, 
                              {'fully_conn_size': (16, 128),
                               'len_layer_conv': (3, 5),
                               'kernel_size': (2, 7),
                               'learning_rate': (0.0001, 1),
                               'pooling_size': (2, 4),
                               'multiply': (1, 3),
                               'dropout_rate': (0.1, 0.99),
                               'beta': (0.000001, 0.49),
                               'activation': (0, 2),
                               'batch_normalization': (0, 1)
                              })
NN_BAYESIAN.maximize(init_points = 20, n_iter = 40, acq = 'ei', xi = 0.0)

我猜你应该为你的模型自定义参数，然后像我上面展示的那样运行贝叶斯优化。祝你好运，如果您有任何问题，请告诉我。