Keras 连接模型时出现无梯度错误

Question

我正在尝试使用带有分层 RNN 模型的 Keras 实现视觉叙事模型，基本上是神经图像字幕样式，但在解码器 RNN 之上使用双向 RNN 的一系列照片。

我分别实现并测试了这个模型的三个部分，CNN、BRNN 和解码器 RNN，但在尝试连接它们时出现此错误：

ValueError: 一个操作没有梯度。请确保您的所有操作都定义了渐变（即可微分）。无梯度的常用操作：K.argmax、K.round、K.eval。

我的代码如下：

#vgg16 model with the fc2 layer as output
cnn_base_model = self.cnn_model.base_model
brnn_model = self.brnn_model.model
rnn_model = self.rnn_model.model

cnn_part = TimeDistributed(cnn_base_model)

img_input = Input((self.story_length,) + self.cnn_model.input_shape, name='brnn_img_input')

extracted_feature = cnn_part(img_input)

#[None, 5, 512], a 512 length vector for each picture in the story
brnn_feature = brnn_model(extracted_feature)

#[None, 5, 25], input groundtruth word indices fed as input when training
decoder_input = Input((self.story_length, self.max_length), name='brnn_decoder_input')

decoder_outputs = []

for i in range(self.story_length):
  #separate timesteps for decoding
  decoder_input_i = Lambda(lambda x: x[:, i, :])(decoder_input)
  brnn_feature_i = Lambda(lambda x: x[:, i, :])(brnn_feature)

  #the problem persists when using Dense instead of the Lambda layers above
  #decoder_input_i = Dense(25)(Reshape((125,))(decoder_input))
  #brnn_feature_i = Dense(512)(Reshape((5 * 512,))(brnn_feature))

  decoder_output_i = rnn_model([decoder_input_i, brnn_feature_i])
  decoder_outputs.append(decoder_output_i)

decoder_output = Concatenate(axis=-2, name='brnn_decoder_output')(decoder_outputs)

self.model = Model([img_input, decoder_input], decoder_output)

以及 BRNN 的代码：

image_feature = Input(shape=(self.story_length, self.img_feature_dim,))
image_emb = TimeDistributed(Dense(self.lstm_size))(image_feature)

brnn = Bidirectional(LSTM(self.lstm_size, return_sequences=True), merge_mode='concat')(image_emb)
brnn_emb = TimeDistributed(Dense(self.lstm_size))(brnn)

self.model = Model(inputs=image_feature, outputs=brnn_emb)

还有RNN：

#[None, 512], the vector to be decoded
initial_input = Input(shape=(self.input_dim,), name='rnn_initial_input')

#[None, 25], the groundtruth word indices fed as input when training 
decoder_inputs = Input(shape=(None,), name='rnn_decoder_inputs')

decoder_input_masking = Masking(mask_value=0.0)(decoder_inputs)
decoder_input_embeddings = Embedding(self.vocabulary_size, self.emb_size,
  embeddings_regularizer=l2(regularizer))(decoder_input_masking)
decoder_input_dropout = Dropout(.5)(decoder_input_embeddings)

initial_emb = Dense(self.emb_size,
  kernel_regularizer=l2(regularizer))(initial_input)

initial_reshape = Reshape((1, self.emb_size))(initial_emb)
initial_masking = Masking(mask_value=0.0)(initial_reshape)
initial_dropout = Dropout(.5)(initial_masking)

decoder_lstm = LSTM(self.hidden_dim, return_sequences=True, return_state=True,
  recurrent_regularizer=l2(regularizer),
  kernel_regularizer=l2(regularizer),
  bias_regularizer=l2(regularizer))

_, initial_hidden_h, initial_hidden_c = decoder_lstm(initial_dropout)

decoder_outputs, decoder_state_h, decoder_state_c = decoder_lstm(decoder_input_dropout,
  initial_state=[initial_hidden_h, initial_hidden_c])

decoder_output_dense_layer = TimeDistributed(Dense(self.vocabulary_size, activation='softmax',
  kernel_regularizer=l2(regularizer)))

decoder_output_dense = decoder_output_dense_layer(decoder_outputs)

self.model = Model([decoder_inputs, initial_input], decoder_output_dense)

我使用adam 作为优化器，sparse_categorical_crossentropy 作为损失。

起初我认为问题在于用于分割时间步的Lambda 层，但当我用Dense 层替换它们时问题仍然存在（这是保证

Answer 1

我遇到了类似的错误，结果我想在 init() 中构建图层（在我的自定义图层或模型中），如下所示：

self.lstm_custom_1 = keras.layers.LSTM(128,batch_input_shape=batch_input_shape, return_sequences=False,stateful=True)
self.lstm_custom_1.build(batch_input_shape)

self.dense_custom_1 = keras.layers.Dense(32,  activation = 'relu')
self.dense_custom_1.build(input_shape=(batch_size, 128))```

Answer 2

我认为问题实际上在于嵌入层。渐变不能通过嵌入层，因此除非它是模型中的第一层，否则它不会起作用。