Tensorflow 教程中的解码器尺寸不匹配

Question

我正在关注 tensorflow 的卷积自动编码器教程，使用 tensorflow 2.0 和 keras，发现 here。

使用提供的代码构建 CNN，但在编码器和解码器中再添加一个卷积层会导致代码中断：

class Denoise(Model):
  def __init__(self):
    super(Denoise, self).__init__()
    self.encoder = tf.keras.Sequential([
      layers.Input(shape=(28, 28, 1)), 
      layers.Conv2D(16, (3,3), activation='relu', padding='same', strides=2),
      layers.Conv2D(8, (3,3), activation='relu', padding='same', strides=2),
      ## New Layer ##
      layers.Conv2D(4, (3,3), activation='relu', padding='same', strides=2)
      ## --------- ##
      ])

    self.decoder = tf.keras.Sequential([
      ## New Layer ##
      layers.Conv2DTranspose(4, kernel_size=3, strides=2, activation='relu', padding='same'),
      ## --------- ##
      layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2D(1, kernel_size=(3,3), activation='sigmoid', padding='same')
      ])

  def call(self, x):
    encoded = self.encoder(x)
    decoded = self.decoder(encoded)
    return decoded

autoencoder = Denoise()

运行autoencoder.encoder.summary() 和autoencoder.decoder.summary()，我可以看到这是一个形状问题：

Encoder:
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_124 (Conv2D)          (None, 14, 14, 16)        160       
_________________________________________________________________
conv2d_125 (Conv2D)          (None, 7, 7, 8)           1160      
_________________________________________________________________
conv2d_126 (Conv2D)          (None, 4, 4, 4)           292       
=================================================================
Total params: 1,612
Trainable params: 1,612
Non-trainable params: 0
_________________________________________________________________

Decoder:
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_transpose_77 (Conv2DT (32, 8, 8, 4)             148       
_________________________________________________________________
conv2d_transpose_78 (Conv2DT (32, 16, 16, 8)           296       
_________________________________________________________________
conv2d_transpose_79 (Conv2DT (32, 32, 32, 16)          1168      
_________________________________________________________________
conv2d_127 (Conv2D)          (32, 32, 32, 1)           145       
=================================================================
Total params: 1,757
Trainable params: 1,757
Non-trainable params: 0
_________________________________________________________________

为什么解码端的前导维度是32？如果输入是从编码器传递的，为什么输入层的维度不是None, 4, 4, 4？我该如何解决这个问题？

提前感谢您对此的帮助！

Answer 1

在最后一个编码器层中删除stride=2，并在最后一个解码器层中添加stride=2。

from tensorflow.keras import layers
from tensorflow.keras import Model

class Denoise(Model):
  def __init__(self):
    super(Denoise, self).__init__()
    self.encoder = tf.keras.Sequential([
      layers.Input(shape=(28, 28, 1)), 
      layers.Conv2D(16, (3,3), activation='relu', padding='same', strides=2),
      layers.Conv2D(8, (3,3), activation='relu', padding='same', strides=2),
      ## New Layer ##
      layers.Conv2D(4, (3,3), activation='relu', padding='same')
      ## --------- ##
      ])

    self.decoder = tf.keras.Sequential([
      ## New Layer ##
      layers.Conv2DTranspose(4, kernel_size=3, strides=2, activation='relu', padding='same'),
      ## --------- ##
      layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2D(1, kernel_size=(3,3), activation='sigmoid', padding='same', strides=2)
      ])

  def call(self, x):
    encoded = self.encoder(x)
    decoded = self.decoder(encoded)
    return decoded

autoencoder = Denoise()
autoencoder.build(input_shape=(1, 28, 28, 1))
autoencoder.summary()

Answer 2

Keras 使用 32 作为默认的 batch_size。这可能与此有关。但要解决此问题，您可以在解码器中包含 input_shape 参数。

self.decoder = tf.keras.models.Sequential([
      ## New Layer ##
      layers.Conv2DTranspose(4, kernel_size=3, strides=2, activation='relu', padding='same', input_shape=(4,4,4)),
      ## --------- ##
      layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2D(1, kernel_size=(3,3), activation='sigmoid', padding='same')
      ])

另外，为了避免潜在的问题和一致性，我将tf.keras.models 用于模型（而不是tf.keras.Model）和tf.keras.layers 用于层。这可能不是问题..但可能会导致问题。