Python：为神经网络定义网格搜索参数的问题

Question

我正在尝试使用以下代码对神经网络的多个参数执行网格搜索：

def create_network(optimizer='rmsprop'):
    
    # Start Artificial Neural Network
    network = Sequential()
    
    # Adding the input layer and the first hidden layer
    # units = neurons
    network.add(Dense(units = 16, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the second hidden layer
    network.add(Dense(units = 16, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the third hidden layer
    network.add(Dense(units = 16, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the output layer
    network.add(Dense(units = 1))
    
    # Compile NN
    network.compile(optimizer = optimizer, 
                loss = 'mean_squared_error', 
                metrics=['mae', tf.keras.metrics.RootMeanSquaredError()])
    
    # Return compiled network
    return network

# Wrap Keras model so it can be used by scikit-learn
ann = KerasRegressor(build_fn=create_network, verbose=0)

# Create hyperparameter space
epoch_values = [10, 25, 50, 100, 150, 200]
batches = [10, 20, 30, 40, 50, 100, 1000]
optimizers = ['rmsprop', 'adam', 'SGD']
neurons = [16, 32, 64, 128, 256]
lr_values = [0.001, 0.01, 0.1, 0.2, 0.3]

# Create hyperparameter options
hyperparameters = dict(optimizer=optimizers, epochs=epoch_values, batch_size=batches, units=neurons,learning_rate=lr_values)

# Create grid search
# cv=5 is the default 5-fold
grid = GridSearchCV(estimator=ann, cv=5, param_grid=hyperparameters)

# Fit grid search
grid_result = grid.fit(X, y)

但我得到了错误：

learning_rate is not a legal parameter

只有优化器、epochs 和 batch_size 起作用……其他参数在搜索中无法识别。

我该如何解决这个问题？

如果有相关了解，我还想在网格搜索中添加更多参数。

Answer 1

目前，您并未指示网络使用学习率，因此scikit-learn 网格搜索不知道如何更改它。明确告诉优化器如何在您的 create_network 函数中更改学习率（neurons 或任何其他参数也是如此）。这样的事情应该可以工作：

def create_network(optimizer='rmsprop', neurons=16, learning_rate=0.001):
    
    # Start Artificial Neural Network
    network = Sequential()
    
    # Adding the input layer and the first hidden layer
    network.add(Dense(units = neurons, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the second hidden layer
    network.add(Dense(units = neurons, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the third hidden layer
    network.add(Dense(units = neurons, 
                  activation = tf.keras.layers.LeakyReLU(alpha=0.3)))

    # Adding the output layer
    network.add(Dense(units = 1))

    ###############################################
    # Add optimizer with learning rate
    if optimizer == 'rmsprop':
        opt = tf.keras.optimizers.RMSprop(learning_rate=learning_rate)
    elif optimizer == 'adam':
        opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
    elif optimizer == 'SGD':
        opt = tf.keras.optimizers.SGD(learning_rate=learning_rate)
    else:
        raise ValueError('optimizer {} unrecognized'.format(optimizer))
    ##############################################    

    # Compile NN
    network.compile(optimizer = opt, 
                loss = 'mean_squared_error', 
                metrics=['mae', tf.keras.metrics.RootMeanSquaredError()])
    
    # Return compiled network
    return network

# Wrap Keras model so it can be used by scikit-learn
ann = KerasRegressor(build_fn=create_network, verbose=0)

# Create hyperparameter space
epoch_values = [10, 25, 50, 100, 150, 200]
batches = [10, 20, 30, 40, 50, 100, 1000]
optimizers = ['rmsprop', 'adam', 'SGD']
neuron_list = [16, 32, 64, 128, 256]
lr_values = [0.001, 0.01, 0.1, 0.2, 0.3]

# Create hyperparameter options
hyperparameters = dict(
    epochs=epoch_values, 
    batch_size=batches, 
    optimizer=optimizers, 
    neurons=neuron_list,
    learning_rate=lr_values)

# Create grid search
# cv=5 is the default 5-fold
grid = GridSearchCV(estimator=ann, cv=5, param_grid=hyperparameters)

# Fit grid search
grid_result = grid.fit(X, y)

也可以对neurons 或与网络结构相关的任何其他参数进行类似的修改。确保将create_network 的参数名称与hyperparameters 中的键匹配。