Keras（FIT_GENERATOR）-检查目标时出错：预期activation_1具有3维，但得到的数组形状为（32、416、608、3）答案

【问题标题】：Keras(FIT_GENERATOR)- Error, when checking target: expected activation_1 to have 3 dimensions, but got array with shape (32, 416, 608, 3)Keras（FIT_GENERATOR）-检查目标时出错：预期activation_1具有3维，但得到的数组形状为（32、416、608、3）
【发布时间】：2018-04-15 16:42:29
【问题描述】：

我已经研究了很多天的分割问题，在终于找到了如何正确读取数据集之后，我遇到了这个问题：

ValueError: Error when checking target: expected activation_1(Softmax) to have 3 dimensions, but got array with shape

(32, 416, 608, 3)

I used the functional API, since I took the FCNN architecture from [here](https://github.com/divamgupta/image-segmentation-keras/blob/master/Models/FCN32.py).

根据我的任务（IMAGE_ORDERING = "channels_last"(TensorFlow backend)）稍作修改和适配。谁能帮帮我吗？提前致谢。下面的架构是针对 FCNN 的，我尝试实现它以用于分割。这是架构（在调用 model.summary() 之后）：

1。

2。

具体错误是：
“导入数据集”功能：

“Fit_Generator 方法调用”：

 img_input = Input(shape=(input_height,input_width,3))

 #Block 1
 x = Convolution2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1', data_format=IMAGE_ORDERING)(img_input) 
 x = BatchNormalization()(x)
 x = Convolution2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool', data_format=IMAGE_ORDERING)(x)
 f1 = x
 # Block 2
 x = Convolution2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool', data_format=IMAGE_ORDERING )(x)
 f2 = x

 # Block 3
 x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool', data_format=IMAGE_ORDERING )(x)
 f3 = x

 # Block 4
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2',data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3',data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool', data_format=IMAGE_ORDERING)(x)
 f4 = x

 # Block 5
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2',data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3', data_format=IMAGE_ORDERING)(x)
 x = BatchNormalization()(x)
 x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool', data_format=IMAGE_ORDERING)(x)
 f5 = x

 x = (Convolution2D(4096,(7,7) , activation='relu' , padding='same', data_format=IMAGE_ORDERING))(x)
 x = Dropout(0.5)(x)
 x = (Convolution2D(4096,(1,1) , activation='relu' , padding='same',data_format=IMAGE_ORDERING))(x)
 x = Dropout(0.5)(x)

 #First parameter = number of classes+1 (de la background)
 x = (Convolution2D(20,(1,1) ,kernel_initializer='he_normal' ,data_format=IMAGE_ORDERING))(x)
 x = Convolution2DTranspose(20,kernel_size=(64,64), strides=(32,32),use_bias=False,data_format=IMAGE_ORDERING)(x)
 o_shape = Model(img_input,x).output_shape

 outputHeight = o_shape[1]
 print('Output Height is:', outputHeight)
 outputWidth = o_shape[2]
 print('Output Width is:', outputWidth)
 #https://keras.io/layers/core/#reshape
 x = (Reshape((20,outputHeight*outputWidth)))(x)
 #https://keras.io/layers/core/#permute
 x = (Permute((2, 1)))(x)
 print("Output shape before softmax is", o_shape)
 x = (Activation('softmax'))(x)
 print("Output shape after softmax is", o_shape)
 model = Model(inputs = img_input,outputs = x)
 model.outputWidth = outputWidth
 model.outputHeight = outputHeight
 model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics =['accuracy'])

【问题讨论】：

错误发生在fit还是compile？
错误发生在“fit_generator”（我忘了说明我使用了Keras的imageDataGenerator和fit_generator方法）。
这意味着你的 y 不是正确的形状。你的 y 是什么形状的？
您的 y 必须是形状 6、6、512，或者您可以添加图层以将模型输出重塑为 192、192、3
是的，谢谢 Primusa 的及时回复。这是代码 sn-p（它不允许我在这里添加，所以请检查帖子）

标签： python keras keras-layer

【解决方案1】：

FCNN 架构示例中的原始代码使用(416, 608) 的输入维度。而在您的代码中，输入维度是(192, 192)（忽略通道维度）。现在，如果你仔细注意，这个特殊的层

x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool', data_format=IMAGE_ORDERING)(x)

生成维度为(6, 6) 的输出（您可以在model.summary() 中验证）。

下一个卷积层

o = (Convolution2D(4096,(7,7) , activation='relu' , padding='same', data_format=IMAGE_ORDERING))(o)

使用大小为(7, 7) 的卷积过滤器，但您的输入已经减小到小于该大小的大小（即(6, 6)）。先尝试修复它。

此外，如果您查看model.summary() 输出，您会注意到它不包含在 block5_pool 层之后定义的层。其中有一个transposed convolution 层（基本上对您的输入进行上采样）。您可能也想看看并尝试解决这个问题。

注意：在我的所有维度中，我都忽略了渠道维度。

编辑下面的详细答案

首先，这是我的keras.json 文件。它使用 Tensorflow 后端，image_ordering 设置在 channel_last。

{
    "floatx": "float32",
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "image_data_format": "channels_last"
}

接下来，我复制粘贴我的确切 model 代码。请特别注意下面代码中的内联 cmets。

from keras.models import *
from keras.layers import *

IMAGE_ORDERING = 'channels_last' # In consistency with the json file

def getFCN32(nb_classes = 20, input_height = 416, input_width = 608):

    img_input = Input(shape=(input_height,input_width, 3)) # Expected input will have channel in the last dimension

    #Block 1
    x = Convolution2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1', data_format=IMAGE_ORDERING)(img_input) 
    x = BatchNormalization()(x)
    x = Convolution2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool', data_format=IMAGE_ORDERING)(x)
    f1 = x
    # Block 2
    x = Convolution2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool', data_format=IMAGE_ORDERING )(x)
    f2 = x

    # Block 3
    x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool', data_format=IMAGE_ORDERING )(x)
    f3 = x

    # Block 4
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2',data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3',data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool', data_format=IMAGE_ORDERING)(x)
    f4 = x

    # Block 5
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2',data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = Convolution2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3', data_format=IMAGE_ORDERING)(x)
    x = BatchNormalization()(x)
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool', data_format=IMAGE_ORDERING)(x)
    f5 = x

    x = (Convolution2D(4096,(7,7) , activation='relu' , padding='same', data_format=IMAGE_ORDERING))(x)
    x = Dropout(0.5)(x)
    x = (Convolution2D(4096,(1,1) , activation='relu' , padding='same',data_format=IMAGE_ORDERING))(x)
    x = Dropout(0.5)(x)

    x = (Convolution2D(20,(1,1) ,kernel_initializer='he_normal' ,data_format=IMAGE_ORDERING))(x)
    x = Convolution2DTranspose(20,kernel_size=(64,64), strides=(32,32),use_bias=False,data_format=IMAGE_ORDERING)(x)
    o_shape = Model(img_input, x).output_shape

    # NOTE: Since this is channel last dimension ordering, the height and width dimensions are along [1] and [2], not [2] and [3]
    outputHeight = o_shape[1]
    outputWidth = o_shape[2]

    x = (Reshape((outputHeight*outputWidth, 20)))(x) # Channel should be along the last dimenion of reshape
    # No need of permute layer anymore

    print("Output shape before softmax is", o_shape)
    x = (Activation('softmax'))(x)
    print("Output shape after softmax is", o_shape)
    model = Model(inputs = img_input,outputs = x)
    model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics =['accuracy'])

    return model

model = getFCN32(20)
print(model.summary())

接下来我将提供我的model.summary() 外观的 sn-ps。如果你看一下最后几层，它是这样的：

所以这意味着，conv2d_transpose 层会产生一个维度为 (448, 640, 20) 的输出，并在对其应用 softmax 之前将其展平。所以输出的维度是(286720, 20)。同样，您的target_generator（在您的情况下为mask_generator）也应该生成类似尺寸的目标。同样，您的 input_generator 也应该生成大小为 [batch size, input_height,input_width, 3] 的输入批次，如您的函数的 img_input 中所述。

希望这将帮助您深入了解问题并找出合适的解决方案。请查看代码中的细微变化（以及内嵌 cmets）以及如何创建 input 和 target 批处理。

【讨论】：

显然代码行 o=f5 没有正常工作，所以是的，你在这方面是对的。
我在每一步都用 (x) 替换了所有的函数调用，但现在又遇到了另一个错误
新的错误是什么？您是否还注意过您的输入尺寸，使其不会减小到小于 (7, 7) 的尺寸？
编写自己的生成器函数。一些指针如下：github.com/keras-team/keras/issues/8078#issuecomment-334909314 和stackoverflow.com/questions/46493419/…。因为无论如何您输出的是 20 通道图像，因此仅使用原生 Keras 图像预处理功能可能不是一个好主意，因为它们仅适用于常规（RGB 或灰度）图像。您需要编写自己的生成器函数来加载目标 20 通道 GT 图像。
看看这个。这是用于生成分割输入和 GT 的生成器。这可能对您有用。 github.com/divamgupta/image-segmentation-keras/blob/master/…

【解决方案2】：

您可能在flow_from_directory() 调用掩码中缺少color_mode='grayscale'。 RGB 是color_mode 的默认值。

flow_args = dict(
    batch_size=batch_size,
    target_size=target_size,
    class_mode=None,
    seed=seed)

image_generator = image_datagen.flow_from_directory(
    image_dir, subset='training', **flow_args)

mask_generator = mask_datagen.flow_from_directory(
    mask_dir, subset='training', color_mode='grayscale', **flow_args)

【讨论】：

【解决方案3】：

我尝试使用 SegNet 架构，但又遇到了完全相同的错误。看来这不是架构问题，而是来自 fit_generator && 使用掩码的问题。

更新：通过将正确形式的输入掩码提供给神经网络来解决问题。

【讨论】：