convert_to_generator_like num_samples 属性错误：“int”对象没有属性“shape”

Question

我已经使用 Keras 序列编写了一个自定义生成器，但是在第一个 epoch 结束时，我得到了： 属性错误：自定义生成器对象没有属性“形状”

Ubuntu 18.04 库达 10 尝试过 TensorFlow 1.13 和 1.14 看到这个页面： https://github.com/keras-team/keras/issues/12586 我试着改变 从 keras.utils 导入序列 到 从 tensorflow.python.keras.utils.data_utils 导入序列 但没有运气！

class CustomGenerator(Sequence):

def __init__(self, ....):
    ...
    # Preallocate memory
    if mode == 'train' and self.crop_shape:
        self.X = np.zeros((batch_size, crop_shape[0], crop_shape[1], 4), dtype='float32')
        # edge
        # self.X2 = np.zeros((batch_size, crop_shape[1], crop_shape[0], 3), dtype='float32')

        self.Y1 = np.zeros((batch_size, crop_shape[0] // 4, crop_shape[1] // 4, self.n_classes), dtype='float32')

def on_epoch_end(self):
    # Shuffle dataset for next epoch
    c = list(zip(self.image_path_list, self.label_path_list, self.edge_path_list))
    random.shuffle(c)
    self.image_path_list, self.label_path_list, self.edge_path_list = zip(*c)

    # Fix memory leak (tensorflow.python.keras bug)
    gc.collect()


def __getitem__(self, index):
    for n, (image_path, label_path,edge_path) in enumerate(
            zip(self.image_path_list[index * self.batch_size:(index + 1) * self.batch_size],
                self.label_path_list[index * self.batch_size:(index + 1) * self.batch_size],
                self.edge_path_list[index * self.batch_size:(index + 1) * self.batch_size])):

        image = cv2.imread(image_path, 1)
        label = cv2.imread(label_path, 0)

        edge = cv2.imread(edge_path, 0)

        ....

        self.X[n] = image
        self.Y1[n] = to_categorical(cv2.resize(label, (label.shape[1] // 4, label.shape[0] // 4)),
                                    self.n_classes).reshape((label.shape[0] // 4, label.shape[1] // 4, -1))
        self.Y2[n] = to_categorical(cv2.resize(label, (label.shape[1] // 8, label.shape[0] // 8)),
                                    self.n_classes).reshape((label.shape[0] // 8, label.shape[1] // 8, -1))
        self.Y3[n] = to_categorical(cv2.resize(label, (label.shape[1] // 16, label.shape[0] // 16)),
                                    self.n_classes).reshape((label.shape[0] // 16, label.shape[1] // 16, -1))

    return self.X, [self.Y1, self.Y2, self.Y3]

def __len__(self):
    return math.floor(len(self.image_path_list) / self.batch_size)

def random_crop(image, edge, label, random_crop_size=(800, 1600)):
    ....
    return image, label

错误是：

742/743 [============================>.] - ETA: 0s - loss: 1.8465 - conv6_cls_loss: 1.1261 - sub24_out_loss: 1.2478 - sub4_out_loss: 1.3827 - conv6_cls_categorical_accuracy: 0.6705 - sub24_out_categorical_accuracy: 0.6250 - sub4_out_categorical_accuracy: 0.5963Traceback (most recent call last):
  File "/home/user/Desktop/Keras-ICNet/train1.py", line 75, in <module>
    use_multiprocessing=True, shuffle=True, max_queue_size=10, initial_epoch=opt.epoch)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1433, in fit_generator
    steps_name='steps_per_epoch')
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_generator.py", line 322, in model_iteration
    steps_name='validation_steps')
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_generator.py", line 144, in model_iteration
    shuffle=shuffle)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_generator.py", line 480, in convert_to_generator_like
    num_samples = int(nest.flatten(data)[0].shape[0])
AttributeError: 'int' object has no attribute 'shape'

Answer 1

查看堆栈跟踪，

num_samples = int(nest.flatten(data)[0].shape[0])
AttributeError: 'int' object has no attribute 'shape'

data实际上是指fit_generator中传入的validation_data参数。这应该是 generator 或 tuple。我的猜测是这是作为数组传递的，因此 nest.flatten(data)[0] 返回 int 并因此返回错误。