Keras ImageDataGenerator 验证精度低

Question

我正在尝试使用 mobilenetv2 进行迁移学习，以对 stanford 的 cars-196 数据集中的 196 类汽车进行分类。

我的工作环境是 google colab notebook。 我使用 keras 的 ImageDataGenerator 加载图像以进行训练和验证。 在训练图像上，我还执行数据增强。

以下代码是我的执行方式：

# To load the dataset from the drive
from google.colab import drive
drive.mount('/content/drive')

import math
from keras.models import Sequential, Model
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout, ReLU, GlobalAveragePooling2D
from keras.preprocessing.image import ImageDataGenerator
from keras.applications.mobilenet_v2 import MobileNetV2, preprocess_input

BATCH_SIZE = 196

train_datagen = ImageDataGenerator(
    rotation_range=20,  # Rotate the augmented image by 20 degrees
    zoom_range=0.2,  # Zoom by 20% more or less
    horizontal_flip=True,  # Allow for horizontal flips of augmented images
    brightness_range=[0.8, 1.2],  # Lighter and darker images by 20%
    width_shift_range=0.1,
    height_shift_range=0.1,
    preprocessing_function=preprocess_input
)


img_data_iterator = train_datagen.flow_from_directory(
    # Where to take the data from, the classes are the sub folder names
    '/content/drive/My Drive/Datasets/cars-196/car_data/train',
    class_mode="categorical",  # classes are in 2D one hot encoded way, default is true but just to point it out
    shuffle=True,  # shuffle the data, default is true but just to point it out
    batch_size=BATCH_SIZE,
    target_size=(224, 224)  # This size is the default of mobilenet NN
)

validation_img_data_iterator = ImageDataGenerator().flow_from_directory(
    '/content/drive/My Drive/Datasets/cars-196/car_data/test',
    class_mode="categorical",
    shuffle=True,
    batch_size=BATCH_SIZE,
    target_size=(224, 224)
)

base_model = MobileNetV2(weights='imagenet', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x)
preds = Dense(196, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=preds)

# Disable training of already trained layer
for layer in model.layers[:-3]:
    layer.trainable = False

model.compile(optimizer='Adam',loss='categorical_crossentropy',metrics=['accuracy'])

# define the checkpoint
from keras.callbacks import ModelCheckpoint
filepath = "/content/drive/My Drive/Datasets/cars-196/model.h5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]

history = model.fit(
    img_data_iterator,
    steps_per_epoch=math.ceil(8144/BATCH_SIZE),  # 8144 is the number of training images
    validation_steps=math.ceil(8062/BATCH_SIZE),  # 8062 is the number of validation images
    validation_data=validation_img_data_iterator,
    epochs=100,
    callbacks=callbacks_list
)

关于批量大小，从this stackoverflow question我决定将批量大小设置为可用标签的数量，但在val_accuracy方面没有任何改变。

我在我添加的全连接层之间添加了 0.5 的 dropout，但同样，验证的准确性没有变化。

我在训练集上的准确率达到 92% 左右，而验证准确率保持在 0.7% 左右。

我的猜测是 ImageDataGenerator 行为怪异并搞砸了准确性，但我还没有找到任何解决问题的方法，所以 ATM 我不知道它背后的原因是什么。

有没有人猜测可能是什么问题？

----- 编辑

train 和 test 文件夹都有带有标签名称的子文件夹（我想识别的不同汽车），每个子文件夹都有那辆车的图像。这就是汽车 196 数据集的样子。 ImageDataGenerator 将正确的标签附加到图像上，具体取决于该图像所在的子文件夹。

Answer 1

问题是我没有将preprocess_input 函数应用于验证数据图像生成器。

而不是

validation_img_data_iterator = ImageDataGenerator().flow_from_directory(
    '/content/drive/My Drive/Datasets/cars-196/car_data/test',
    class_mode="categorical",
    shuffle=True,
    batch_size=BATCH_SIZE,
    target_size=(224, 224)
)

改成

validation_img_data_iterator = ImageDataGenerator(
    preprocessing_function=preprocess_input
).flow_from_directory(
    '/content/drive/My Drive/Datasets/cars-196/car_data/test',
    class_mode="categorical",
    shuffle=True,
    batch_size=BATCH_SIZE,
    target_size=(224, 224)
)