急切执行函数的输入不能是带有变分自动编码器的 Keras 符号张量

Question

我正在尝试实现自定义变分自动编码器。代码如下所示

image = Input(shape = (X_train.shape[1]))
label = Input(shape = (Y_train.shape[1]))

inputs = Concatenate()([image, label])

x = Dense(625, activation = 'relu')(inputs)
x = Reshape((25,25,1))(x)

x = LocallyConnected2D(8, (5,5), padding = 'valid')(x)
x = LeakyReLU()(x)

x = LocallyConnected2D(8, (5,5), padding = 'valid')(x)
x = LeakyReLU()(x)

x = LocallyConnected2D(8, (3,3), padding = 'valid')(x)
x = LeakyReLU()(x)

x = LocallyConnected2D(8, (3,3), padding = 'valid')(x)
x = LeakyReLU()(x)

x = AveragePooling2D((2, 2))(x)

encoder_out = Flatten()(x)

mu = Dense(latent_size, activation ='linear')(encoder_out)
sigma = Dense(latent_size, activation = 'linear')(encoder_out)

def sampling(args):
    mu, sigma = args
    eps = K.random_normal(shape=(batch_size, latent_size), mean=0., stddev=1.)
    return mu + K.exp(sigma / 2) * eps

latent_space = Lambda(sampling, output_shape = (latent_size, ))([mu, sigma])

decoder_latent = Input(shape = (latent_size, ))
decoder_c = Input(shape = (c_space, ))

x = Concatenate()([decoder_latent, decoder_c])

x = Dense(288)(x)
x = Reshape((6,6,8))(x)
x = ZeroPadding2D((2,2))(x)
x = LocallyConnected2D(8, (3,3), padding = 'valid')(x)
x = LeakyReLU()(x)

x = ZeroPadding2D((2,2))(x)
x = LocallyConnected2D(8, (3,3), padding = 'valid')(x)
x = LeakyReLU()(x)

x = UpSampling2D(size = (2,2))(x)
x = LocallyConnected2D(8, (5,5), padding = 'valid')(x)
x = LeakyReLU()(x)

x = UpSampling2D(size = (2,2))(x)
x = LocallyConnected2D(8,(5,5), padding = 'valid')(x)
x = LeakyReLU()(x)

x = LocallyConnected2D(1,(4,4), padding = 'valid')(x)
decoder_out = Activation('relu')(x)

我定义为的损失函数

def DFC_loss(x_in, x_out):
    kl_loss = 0.5 * K.sum(K.exp(sigma) + K.square(mu) - 1. - sigma, axis=1)
    return K.mean(perceptual_loss(x_in, x_out) + kl_loss)

def perceptual_loss(x_in, x_out):
    
    x_in = K.reshape(x_in, shape=(batch_size, 25,25,1))
    x_out = K.reshape(x_out, shape=(batch_size, 25,25,1))
    
    conv_outputs = [classifier.get_layer(l).output for l in selected_layers]
    
    activation = Model(classifier.input, conv_outputs)

    h1_list = activation(x_in)
    h2_list = activation(x_out)
    
    rc_loss = 0.0
    
    for h1, h2, weight in zip(h1_list, h2_list, [1.0, 1.0]):
        h1 = K.batch_flatten(h1)
        h2 = K.batch_flatten(h2)
        rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
    
    return rc_loss

CVAE.compile(optimizer = "adam", loss = DFC_loss, metrics = [perceptual_loss])

每当我运行下面的代码时

CVAE_hist = CVAE.fit([X_train,Y_train], X_train, verbose = 1, batch_size=batch_size, epochs=n_epochs, validation_data = ([X_test, Y_test], X_test))

我得到两个错误

An op outside of the function building code is being passed
a "Graph" tensor. It is possible to have Graph tensors
leak out of the function building context by including a
tf.init_scope in your function building code.
For example, the following function will fail:
  @tf.function
  def has_init_scope():
    my_constant = tf.constant(1.)
    with tf.init_scope():
      added = my_constant * 2
The graph tensor has name: dense_2_1/Identity:0

和

Inputs to eager execution function cannot be Keras symbolic tensors, but found [<tf.Tensor 'dense_2_1/Identity:0' shape=(None, 6) dtype=float32>, <tf.Tensor 'dense_1_1/Identity:0' shape=(None, 6) dtype=float32>]

有趣的是，每当我将损失函数设置为仅没有 Kl 散度损失的感知损失时，我的代码都没有收到错误。变分自动编码器的 KL 散度损失有很多实现，但我不知道为什么它不适用于这个特定的实现。

Answer 1

我遇到同样的问题很长时间了，但设法解决了。 问题是 TF 只接受接受 (input, output) 参数的损失函数，然后进行比较。但是，您还使用 mu 和 sigma 计算您的 (kl_) 损失，它们基本上是密集层。在 tensorflow v2.1 之前，它神奇地知道这些参数是什么，并且知道如何包含/操作它们，但从那时起，你必须更加小心。阅读this tutorial 后（编辑：也滚动到页面底部以查看完整的 VAE 示例）我建议对您的代码进行以下更改：

1.编译模型时，只定义perceptual_loss 为损失：

CVAE.compile(optimizer = "adam", loss = perceptual_loss, metrics = [perceptual_loss])

2。将sampling 函数更改为一个类，并在call 方法下，添加您的kl_loss，类似

class Sampling(keras.layers.Layer):
    def __init__(self):
        super(Sampling, self).__init__()

    def build(self, input_shape):
        _, sigma_shape = input_shape
        self.sigma_shape = (sigma_shape[-1], )

    def call(self, inputs):
        mu, sigma = inputs

        # Add loss
        kl_loss = 0.5 * K.sum(K.exp(sigma) + K.square(mu) - 1. - sigma, axis=1)
        self.add_loss(K.mean(kl_loss))
        
        # Return sampling as before
        eps = K.random_normal(self.sigma_shape, mean=0., stddev=1.)
        return mu + K.exp(sigma / 2) * eps

确保您只添加kl_loss 而不是DFC_loss ，这样您就不会计算两次perceptual_loss。

3。使用Sampling 类作为潜在层

latent_space = Sampling()([mu, sigma])

这是我的第一个答案，希望对你有帮助！

附：也许您也可以尝试通过add_loss 命令在采样类中移动这两个损失，然后在没有任何loss 参数的情况下进行编译。