Keras LSTM VAE 无效的输出形状

Question

我正在尝试实现基于 LSTM 的 VAE。 输入形状为 (sample_number, 96, 24) 我希望模型的输出形状为 (24)

# encoder
latent_dim = 24
inter_dim = 32
timesteps, features = 96, 24

def sampling(args):
    z_mean, z_log_sigma = args
    batch_size = tf.shape(z_mean)[0] # <================
    epsilon = K.random_normal(shape=(batch_size, latent_dim), mean=0., stddev=1.)
    return z_mean + z_log_sigma * epsilon

# timesteps, features
input_x = Input(shape= (timesteps, features)) 

#intermediate dimension 
h = LSTM(inter_dim)(input_x)


#z_layer
z_mean = Dense(latent_dim)(h)
z_log_sigma = Dense(latent_dim)(h)
z = Lambda(sampling)([z_mean, z_log_sigma])

# Reconstruction decoder
decoder1 = RepeatVector(timesteps)(z)
decoder1 = LSTM(inter_dim, return_sequences=True)(decoder1)
decoder1 = Dense(features)(decoder1)


output = (Dense(24, activation='softmax'))(decoder1)


def vae_loss2(input_x, decoder1, z_log_sigma, z_mean):
    """ Calculate loss = reconstruction loss + KL loss for each data in minibatch """
    # E[log P(X|z)]
    recon = K.sum(K.binary_crossentropy(input_x, decoder1))
    # D_KL(Q(z|X) || P(z|X)); calculate in closed form as both dist. are Gaussian
    kl = 0.5 * K.sum(K.exp(z_log_sigma) + K.square(z_mean) - 1. - z_log_sigma)

    return recon + kl

m = Model(input_x, output)
m.add_loss(vae_loss2(input_x, decoder1, z_log_sigma, z_mean)) #<===========
m.compile(loss=categorical_crossentropy, optimizer='adam', metrics=['accuracy'])

这是目前为止的代码，但是我如何设置这个模型以便它可以有 24 个并在其末尾激活 softmax？

Answer 1

如果您正在训练文本 VAE，您可以尝试使用 TimeDistributed 层，如下所示：

decoder1 = tf.keras.layers.RepeatVector(timesteps)(z)
decoder1 = tf.keras.layers.LSTM(inter_dim, return_sequences=True)(decoder1)
output =  tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(24, activation='linear'))(decoder1)

在你的损失函数中使用SparseCategoricalCrossentropy:

def vae_loss2(input_x, output, z_log_sigma, z_mean):

    cross_entropy = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    recon = cross_entropy(input_x, output))

    kl = 0.5 * K.sum(K.exp(z_log_sigma) + K.square(z_mean) - 1. - z_log_sigma)

    return recon + kl

m = Model(input_x, output)
m.add_loss(vae_loss2(input_x, output, z_log_sigma, z_mean))
m.compile(optimizer='adam', metrics=['accuracy'])

TimeDistributed 层只是将具有 softmax 激活函数的 Dense 层应用于来自LSTM 层的序列中的每个时间步。