使用手动指标使用 hyperas 进行 Keras 超参数调整

Question

我正在使用hyperas document example 调整网络参数，但基于 f1 分数而不是准确度。

我正在使用以下实现 f1 分数：

from keras import backend as K

def f1(y_true, y_pred):
    def recall(y_true, y_pred):
        """Recall metric.
        Only computes a batch-wise average of recall.
        Computes the recall, a metric for multi-label classification of
        how many relevant items are selected.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
        recall = true_positives / (possible_positives + K.epsilon())
        return recall

    def precision(y_true, y_pred):
        """Precision metric.
        Only computes a batch-wise average of precision.
        Computes the precision, a metric for multi-label classification of
        how many selected items are relevant.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
        precision = true_positives / (predicted_positives + K.epsilon())
        return precision
    precision = precision(y_true, y_pred)
    recall = recall(y_true, y_pred)
    return 2*((precision*recall)/(precision+recall+K.epsilon()))

在以下代码行中更新 compile 函数的 metric 参数：

model.compile(loss='categorical_crossentropy', metrics=['accuracy'],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

到

model.compile(loss='categorical_crossentropy', metrics=[f1],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

上述指标在不使用 hyperas 的情况下完美运行，而当我尝试在调整过程中使用它时，我收到以下错误：

Traceback (most recent call last):
  File "D:/path/test.py", line 96, in <module>
    trials=Trials())
  File "C:\Python35\lib\site-packages\hyperas\optim.py", line 67, in minimize
    verbose=verbose)
  File "C:\Python35\lib\site-packages\hyperas\optim.py", line 133, in base_minimizer
    return_argmin=True),
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 367, in fmin
    return_argmin=return_argmin,
  File "C:\Python35\lib\site-packages\hyperopt\base.py", line 635, in fmin
    return_argmin=return_argmin)
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 385, in fmin
    rval.exhaust()
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 244, in exhaust
    self.run(self.max_evals - n_done, block_until_done=self.asynchronous)
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 218, in run
    self.serial_evaluate()
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 137, in serial_evaluate
    result = self.domain.evaluate(spec, ctrl)
  File "C:\Python35\lib\site-packages\hyperopt\base.py", line 840, in evaluate
    rval = self.fn(pyll_rval)
  File "D:\path\temp_model.py", line 86, in keras_fmin_fnct
NameError: name 'f1' is not defined

Answer 1

如果您遵循您链接到的代码示例，您并没有让 hyperas 意识到自定义 f1 函数。包作者也提供了example to do that。

简而言之，您需要在您的 optim.minimize() 调用中添加一个额外的 functions 参数。类似的东西

    best_run, best_model = optim.minimize(model=model,
     data=data,
     functions=[f1],
     algo=tpe.suggest,
     max_evals=5,
     trials=Trials())

实际上我今天刚刚实现了它，所以我相信你也可以让它工作:)