保存卷积神经网络中超参数的优化

Question

在我的卷积神经网络中保存超参数的训练过程时，我遇到了一个问题。我已经阅读了几篇博客文章，但不知何故我无法做到这一点。

我有以下代码：

def ConvNet(embeddings, max_sequence_length, num_words, embedding_dim, trainable=False, extra_conv=True,
            lr=0.0001, dropout=0.7, filters = 128, momentum = 0.8, units = 32, pool_size = 3):
    embedding_layer = Embedding(num_words,
                                embedding_dim,
                                weights=[embeddings],
                                input_length=max_sequence_length,
                                trainable=trainable)

    sequence_input = Input(shape=(max_sequence_length,), dtype='int32')
    embedded_sequences = embedding_layer(sequence_input)
    convs = []
    filter_sizes = [3, 4, 5]
    for filter_size in filter_sizes:
        l_conv = Conv1D(filters=filters, kernel_size=filter_size, activation='relu')(embedded_sequences)
        l_pool = MaxPooling1D(pool_size=pool_size)(l_conv)
        l_conv2 = Conv1D(filters=filters, kernel_size=3, activation='relu')(l_pool)
        l_pool2 = MaxPooling1D(pool_size=pool_size)(l_conv2)

        convs.append(l_pool2)

    l_merge = concatenate(convs, axis=1)

    # add a 1D convnet with global maxpooling, instead of Yoon Kim model
    conv = Conv1D(filters=filters, kernel_size=3, activation='relu')(embedded_sequences)
    pool = MaxPooling1D(pool_size=pool_size)(conv)

    if extra_conv == True:
        x = Dropout(dropout)(l_merge)
    else:
        # Original Yoon Kim model
        x = Dropout(dropout)(pool)
    x = Flatten()(x)
    x = Dense(units = units, activation='relu')(x)
    preds = Dense(1, activation='linear')(x)

    model = Model(sequence_input, preds)
    sgd = keras.optimizers.SGD(learning_rate = lr, momentum= momentum)
    model.compile(loss= r_square_loss,
                  optimizer= sgd,
                  metrics=['mean_squared_error', rmse, r_square])

    model.summary()
    return model 

我正在使用以下函数优化超参数：

from hyperopt import fmin, hp, tpe, space_eval, Trials

def train_and_score(args):
    # Train the model the fixed params plus the optimization args.
    # Note that this method should return the final History object.
    model = ConvNet(embeddings=train_embedding_weights, max_sequence_length= MAX_SEQUENCE_LENGTH,
                    num_words=len(train_word_index)+1, embedding_dim= EMBEDDING_DIM,
                   trainable=False, extra_conv=True,
                   lr=args['lr'], dropout=args['dropout'], filters=args['filters'],
                    momentum= args['momentum'], units = args['units'])
    early_stopping = EarlyStopping(monitor='mean_squared_error', patience=40, verbose=1, mode='auto')

    hist = model.fit(x_train, y_tr, epochs=args['epochs'], batch_size=args['batch_size'], validation_split=0.2, shuffle=True,
                     callbacks=[early_stopping])

    #Unpack and return the last validation loss from the history.
    return hist.history['val_loss'][-1]

 #Define the space to optimize over.
space = {
    'lr': hp.choice('lr', [0.1, 0.01, 0.001, 0.0001]),
    'dropout': hp.choice('dropout', [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]),
    'filters': hp.choice('filters', [32, 64, 128, 256]),
    'pool_size': hp.choice('pool_size', [2, 3]),
    'momentum': hp.choice('momentum', [0.4, 0.5, 0.6, 0.7, 0.8, 0.9]),
    'units': hp.choice('units', [32, 64, 128, 256]),
    'epochs': hp.choice('epochs', [20, 30, 40, 50, 60, 70]),
    'batch_size': hp.choice('batch_size', [20, 30, 40, 50, 60, 70, 80])
}

# Minimize the training score over the space.
trials = Trials()
best = fmin(fn=train_and_score,
            space=space,
            trials=trials,
            max_evals = 10,
            algo=tpe.suggest)

# Print details about the best results and hyperparameters.
print(best)
print(space_eval(space, best))

到目前为止，我的 max_evals 等于 10，看看是否一切正常。对于实际的训练过程，我想将其设置为 500 并让它运行一天... 这是我的问题： 如何保存训练过程？我认为将最好的保存在文件或其他东西中就足够了，因为这是一个大学项目，我必须提交我训练 CNN 的“证明”。

补充问题：截至目前，经过 10 次评估，我正在取最好的参数并手动将其填充到上面提供的代码中，以预测测试集并计算一些统计数字，例如 mse， r-square等。

model = ConvNet(train_embedding_weights, MAX_SEQUENCE_LENGTH, len(train_word_index)+1, EMBEDDING_DIM,
                trainable=False, extra_conv=True,
                lr=0.0001, dropout=0.6, filters= 128,
                momentum= 0.8, units = 32, pool_size = 2)

#define callbacks
early_stopping = EarlyStopping(monitor='mean_squared_error', patience=40, verbose=1, mode='auto')

hist = model.fit(x_train, y_tr, epochs=30, batch_size=20, validation_split=0.2, shuffle=False, callbacks=[early_stopping])

我的梦想是将 max_eval 设置为 500，并将结果存储在输出文件中（最好的超参数组合就足够了），然后自动采用获得的最佳超参数来计算 x测试和统计数字mse、r-square等

有人可以帮忙吗？我被困在这里很多很多很多小时。

谢谢！

Answer 1

我没有这个问题的确切答案，但是有一个here 可能会做到这一点。

它建议在代码末尾打印每个测试的内容。也许您也可以将“试验”对象保存在泡菜或其他东西中，以便您以后自己解析和检查。 我确实有完全相同的问题，我很惊讶没有像 keras 回调那样“简单”的解决方案来保存经过最佳训练的模型。

我还建议你更改问题的标题，至少添加“Hyperopt”、“回调”和“保存模型”关键词。多加注意，也许我们的问题会得到回答:)