图像分类数据生成器答案

【问题标题】：Data generator for image classification图像分类数据生成器
【发布时间】：2022-10-01 07:38:12
【问题描述】：

我正在尝试为我的 CNN 项目创建一个数据生成器（在 keras 中使用顺序模型）。由于数据量很大，我需要不断地将数据流向模型训练，这样我就不会在 RAM 上出现 OOM。但是，我在创建生成器时遇到了一些麻烦。生成器应该接收 batch_size 的数据，然后创建 X 个增强图像。然后我想创建一批创建的增强图像和原始图像，例如 30 张原始图片，每张图像 5 张增强图像 = 30 张原始图片 + 150 张增强图片 = 一批总共 180 张图片。然后我想从这 180 张图片中获取一个 batch_size，比如说 30 张，这将创建 6 个 epoch 步骤，每步 30 张图像。然后我想生成一批新的图像并重复这些步骤 X 数量的时期。

代码：

class customDataGen(tf.keras.utils.Sequence):
    data_holder_x = []
    data_holder_y = []
    
    ## leave out img_gen, that does not do anything right now.
    def __init__(self, X, y, img_gen, batch_size, shuffle = True):
        self.X = X
        self.y = y
        self.batch_size = batch_size
        self.shuffle = shuffle
        self.img_gen = img_gen
        
        nr1 = 5*self.batch_size ## The image augmentation does generates 5 images per image so im just hard-coding in 5 right now.
        nr2 = self.batch_size ## this is the original pictures
        self.n = nr1 + nr2
        self.indices = list(range(0,self.n))
        self.__get_data(index=1) ## just generating a instance of get_data 
        
    
    def on_epoch_end(self):
        self.index = np.arange(len(self.indices))
        if self.shuffle == True:
            np.random.shuffle(self.index)
    
    def __get_data(self,index):
        print(\"get_data startad\")
        aug_img = img_aug(self.X[index*self.batch_size:(index+1)*self.batch_size],self.y[index*self.batch_size:(index+1)*self.batch_size])
        X = list(self.X[index*self.batch_size:(index+1)*self.batch_size])
        y = list(self.y[index*self.batch_size:(index+1)*self.batch_size])                  
        X.extend(aug_img[0])
        y.extend(aug_img[1])
        customDataGen.data_holder_x.append(X)
        customDataGen.data_holder_y.append(y)
    
    def __data_holder(self,index):
        container_x = []
        container_y = []
        print(\"__data_holder startad\")
        if len(customDataGen.data_holder_x[0]) == 0:
            self.__get_data(index)
            container_x.append(customDataGen.data_holder_x[0][:self.batch_size])
            container_y.append(customDataGen.data_holder_y[0][:self.batch_size])
            del customDataGen.data_holder_x[0][:self.batch_size], customDataGen.data_holder_y[0][:self.batch_size]
        else:
            container_x.append(customDataGen.data_holder_x[0][:self.batch_size])
            container_y.append(customDataGen.data_holder_y[0][:self.batch_size])
            del customDataGen.data_holder_x[0][:self.batch_size], customDataGen.data_holder_y[0][:self.batch_size]
        #X = np.array(container_x[0][0])
        #y = np.array(container_y[0][0])
        print(\"remaining data of data_holder_x\", len(customDataGen.data_holder_x[0]))
        return container_x[0],container_y[0]
        
    def __getitem__(self,index):
        container_x,container_y = self.__data_holder(index)
        print(\"get_item startad\")
        X = tf.convert_to_tensor(container_x)
        y = tf.convert_to_tensor(container_y)
        return X,y
    
    def __len__(self):
        return (self.n)//self.batch_size

我现在的问题是似乎 __get_item 被调用并且在 epoch 开始前由 model.fit() 发起 3 次

__data_holder startad
remaining data of data_holder_x 160
get_item startad
Epoch 1/2
__data_holder startad
remaining data of data_holder_x 128
get_item startad
__data_holder startad
remaining data of data_holder_x 96
get_item startad
1/6 [====>.........................] - ETA: 15s - loss: 1.7893 - accuracy: 0.1562__data_holder startad
remaining data of data_holder_x 64
get_item startad
2/6 [=========>....................] - ETA: 6s - loss: 1.7821 - accuracy: 0.2344 __data_holder startad
remaining data of data_holder_x 32
get_item startad
3/6 [==============>...............] - ETA: 4s - loss: 1.7879 - accuracy: 0.1562__data_holder startad
remaining data of data_holder_x 0
get_item startad
4/6 [===================>..........] - ETA: 3s - loss: 1.7878 - accuracy: 0.1953__data_holder startad
get_data startad
remaining data of data_holder_x 0
get_item startad
5/6 [========================>.....] - ETA: 1s - loss: 1.7888 - accuracy: 0.1875

然后出现错误

2022-09-30 17:44:31.255235: W tensorflow/core/framework/op_kernel.cc:1733] INVALID_ARGUMENT: TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.
Traceback (most recent call last):

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/ops/script_ops.py\", line 270, in __call__
    ret = func(*args)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/autograph/impl/api.py\", line 642, in wrapper
    return func(*args, **kwargs)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/data/ops/dataset_ops.py\", line 1073, in generator_py_func
    raise TypeError(

TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.


---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
Input In [298], in <cell line: 1>()
----> 1 model.fit(training,
      2           validation_data=validation,
      3           epochs=2, callbacks = [checkpoint])

File /usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File /usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52 try:
     53   ctx.ensure_initialized()
---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                       inputs, attrs, num_outputs)
     56 except core._NotOkStatusException as e:
     57   if name is not None:

InvalidArgumentError: Graph execution error:

2 root error(s) found.
  (0) INVALID_ARGUMENT:  TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.
Traceback (most recent call last):

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/ops/script_ops.py\", line 270, in __call__
    ret = func(*args)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/autograph/impl/api.py\", line 642, in wrapper
    return func(*args, **kwargs)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/data/ops/dataset_ops.py\", line 1073, in generator_py_func
    raise TypeError(

TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.


     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
     [[IteratorGetNext/_2]]
  (1) INVALID_ARGUMENT:  TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.
Traceback (most recent call last):

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/ops/script_ops.py\", line 270, in __call__
    ret = func(*args)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/autograph/impl/api.py\", line 642, in wrapper
    return func(*args, **kwargs)

  File \"/usr/local/lib/python3.9/dist-packages/tensorflow/python/data/ops/dataset_ops.py\", line 1073, in generator_py_func
    raise TypeError(

TypeError: `generator` yielded an element of shape (0,) where an element of shape (None, None, None, None) was expected.


     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_5083]

我是 python 和 tensorflow 的新手，所以任何帮助表示赞赏。

谢谢，

蟒蛇诺拉

标签： tensorflow conv-neural-network generator image-augmentation

【解决方案1】：

通过数据增强，Sequence 类使用起来并不简单。构建图时似乎调用 getitem前第一个时代。 Sequence 类的一个很好且有趣的替代方案是使用 tensorflow tf.data.Dataset 这是一些可以完成这项工作的虚拟代码：

mydataset = tf.data.Dataset.from_tensor_slices((X, y))
def aug_data(x, y):
  # dummy data augmentation
  x_aug = tf.repeat(x, 5, axis=0)
  y_aug = tf.repeat(y, 5, axis=0)
  # concatenate with original data
  x_aug = tf.concat([x, x_aug], axis=0)
  y_aug = tf.concat([y, y_aug], axis=0)
  return x_aug, y_aug

# batch and apply data augmenatation after
mydataset = mydataset.batch(BATCH_SIZE).map(aug_data)

它将数据扩充分别应用于每个新批次。

【讨论】：