Gridsearch 多层自动编码器答案

【问题标题】：Gridsearch multi layer autoencodersGridsearch 多层自动编码器
【发布时间】：2021-09-07 05:33:59
【问题描述】：

我的模型-

# define encoder
visible = Input(shape=(n_inputs,))
# encoder level 1
e = Dense(400)(visible)
e = Dropout(0.05)(e)
e = ReLU()(e)

# encoder level 2
e = Dense(300)(visible)
e = Dropout(0.05)(e)
e = ReLU()(e)

# encoder level 3
e = Dense(200)(visible)
e = Dropout(0.05)(e)
e = ReLU()(e)

# encoder level 4
e = Dense(100)(visible)
e = Dropout(0.05)(e)
e = ReLU()(e)

# encoder level 4
e = Dense(50)(visible)
e = Dropout(0.05)(e)
e = ReLU()(e)



# bottleneck
n_bottleneck = n_inputs
bottleneck = Dense(n_bottleneck)(e)


# define decoder, level 1
d = Dense(50)(bottleneck)
d = Dropout(0.05)(d)
d = ReLU()(d)

# define decoder, level 2
d = Dense(100)(bottleneck)
d = Dropout(0.05)(d)
d = ReLU()(d)

# define decoder, level 3
d = Dense(200)(bottleneck)
d = Dropout(0.05)(d)
d = ReLU()(d)

# define decoder, level 4
d = Dense(300)(bottleneck)
d = Dropout(0.05)(d)
d = ReLU()(d)

# define decoder, level 4
d = Dense(400)(bottleneck)
d = Dropout(0.05)(d)
d = ReLU()(d)


# output layer
output = Dense(n_inputs, activation='sigmoid')(d)
# define autoencoder model
model = Model(inputs=visible, outputs=output)
# compile autoencoder model
opt = keras.optimizers.Adam(lr=0.00001)
model.compile(optimizer=opt, loss='binary_crossentropy')
# plot the autoencoder
plot_model(model, 'drive/MyDrive/autoencoder_no_compress.png', show_shapes=True)
# fit the autoencoder model to reconstruct input
history = model.fit(X_train_norm, X_train_norm, epochs=500, batch_size=64, verbose=2, validation_split=0.1)

如何使用 gridsearch 来调整我的模型？

基本的网格搜索看起来像这样-

from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import GridSearchCV

def create_model():
    <return a compiled but untrained keras model>

model = KerasClassifier(build_fn = create_model, batch_size=1000, epochs=10)
#now write out all the parameters you want to try out for the grid search
activation = ['relu', 'tanh', 'sigmoid'...]
learn_rate = [0.1, 0.2, ...]
init = ['unform', 'normal', 'zero', ...]
optimizer = ['SGD', 'Adam' ...]
param_grid = dict(activation=activation, learn_rate=learn_rate, init=init, optimizer=optimizer)
grid = GridSearchCV(estimator=model, param_grid=param_grid)
result = grid.fit(X, y)

如何调整它以适合我的模型用例？

【问题讨论】：

标签： python tensorflow machine-learning keras scikit-learn

【解决方案1】：

这是一个为自动编码器调整神经元和 dropout 的示例

定义模型结构：

def create_model(neurons, n_bottleneck, drop_rate):

  def create_blok(x, units, drop_rate=0.05):
    x = Dense(units)(x)
    x = Dropout(drop_rate)(x)
    x = Activation("relu")(x)
    return x
  
  visible = Input(shape=(n_inputs,))
  e = create_blok(visible, int(neurons), drop_rate)
  e = create_blok(e, int(neurons*3/4), drop_rate)
  e = create_blok(e, int(neurons*2/4), drop_rate)
  e = create_blok(e, int(neurons*1/4), drop_rate)
  e = create_blok(e, int(neurons*1/(4*2)), drop_rate)
  bottleneck = Dense(n_bottleneck)(e)
  d = create_blok(bottleneck, int(neurons*1/(4*2)), drop_rate)
  d = create_blok(d, int(neurons*1/4), drop_rate)
  d = create_blok(d, int(neurons*2/4), drop_rate)
  d = create_blok(d, int(neurons*3/4), drop_rate)
  d = create_blok(d, int(neurons), drop_rate)
  output = Dense(n_inputs, activation='sigmoid')(d)

  model = Model(inputs=visible, outputs=output)
  model.compile(optimizer='Adam', loss='binary_crossentropy')

  return model

定义参数：

n_bottleneck = [32, 64, 128]
neurons = [400, 600]
drop_rate = [0.05, 0.2]

param_grid = dict(neurons=neurons, n_bottleneck=n_bottleneck, drop_rate=drop_rate)
param_grid

调整参数：

model = KerasRegressor(build_fn = create_model, batch_size=128, epochs=10, verbose=0)

grid = GridSearchCV(estimator=model, param_grid=param_grid)
result = grid.fit(X, X)

here 正在运行的笔记本

【讨论】：