无法让 tf.train.shuffle_batch() 正常工作

Question

我已经为此工作了一整天，我不认为另一个会有所作为！

我有一个.png 文件，我从中制作了 >400 份[ I got to use images with different shapes, but for now I just want to get this starting ]

这里是我使用带有标签的张量跳到图像的代码：

import tensorflow as tf
import os
import numpy
batch_Size =20
num_epochs = 100
files = os.listdir("Test_PNG")
files = ["Test_PNG/" + s for s in files]
files = [os.path.abspath(s) for s in files ]


def read_my_png_files( filename_queue):
    reader = tf.WholeFileReader()
    imgName,imgTensor = reader.read(filename_queue)
    img =  tf.image.decode_png(imgTensor,channels=0)
    # Processing should be add
    return img,imgName

def inputPipeline(filenames, batch_Size, num_epochs= None):
    filename_queue  = tf.train.string_input_producer(filenames, num_epochs=num_epochs,shuffle =True)
    img_file, label = read_my_png_files(filename_queue)
    min_after_dequeue = 100
    capacity = min_after_dequeue+3*batch_Size
    img_batch,label_batch = tf.train.shuffle_batch([img_file,label],batch_size=batch_Size,enqueue_many=True,
                                                    allow_smaller_final_batch=True, capacity=capacity,
                                                    min_after_dequeue =min_after_dequeue, shapes=[w,h,d])
    return img_batch,label_batch

images, Labels  = inputPipeline(files,batch_Size,num_epochs)

根据我的理解，我应该得到 20 次图像作为张量及其标签。 当我运行下面的代码时，我得到了：

    ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-08857195e465> in <module>()
     34     return img_batch,label_batch
     35 
---> 36 images, Labels  = inputPipeline(files,batch_Size,num_epochs)

<ipython-input-3-08857195e465> in inputPipeline(filenames, batch_Size, num_epochs)
     31     img_batch,label_batch = tf.train.shuffle_batch([img_file,label],batch_size=batch_Size,enqueue_many=True,
     32                                                     allow_smaller_final_batch=True, capacity=capacity,
---> 33                                                     min_after_dequeue =min_after_dequeue, shapes=[w,h,d])
     34     return img_batch,label_batch
     35 

c:\users\engine\appdata\local\programs\python\python35\lib\site-packages\tensorflow\python\training\input.py in shuffle_batch(tensors, batch_size, capacity, min_after_dequeue, num_threads, seed, enqueue_many, shapes, allow_smaller_final_batch, shared_name, name)
   1212       allow_smaller_final_batch=allow_smaller_final_batch,
   1213       shared_name=shared_name,
-> 1214       name=name)
   1215 
   1216 

c:\users\engine\appdata\local\programs\python\python35\lib\site-packages\tensorflow\python\training\input.py in _shuffle_batch(tensors, batch_size, capacity, min_after_dequeue, keep_input, num_threads, seed, enqueue_many, shapes, allow_smaller_final_batch, shared_name, name)
    767     queue = data_flow_ops.RandomShuffleQueue(
    768         capacity=capacity, min_after_dequeue=min_after_dequeue, seed=seed,
--> 769         dtypes=types, shapes=shapes, shared_name=shared_name)
    770     _enqueue(queue, tensor_list, num_threads, enqueue_many, keep_input)
    771     full = (math_ops.cast(math_ops.maximum(0, queue.size() - min_after_dequeue),

c:\users\engine\appdata\local\programs\python\python35\lib\site-packages\tensorflow\python\ops\data_flow_ops.py in __init__(self, capacity, min_after_dequeue, dtypes, shapes, names, seed, shared_name, name)
    626         shared_name=shared_name, name=name)
    627 
--> 628     super(RandomShuffleQueue, self).__init__(dtypes, shapes, names, queue_ref)
    629 
    630 

c:\users\engine\appdata\local\programs\python\python35\lib\site-packages\tensorflow\python\ops\data_flow_ops.py in __init__(self, dtypes, shapes, names, queue_ref)
    151     if shapes is not None:
    152       if len(shapes) != len(dtypes):
--> 153         raise ValueError("Queue shapes must have the same length as dtypes")
    154       self._shapes = [tensor_shape.TensorShape(s) for s in shapes]
    155     else:

ValueError: Queue shapes must have the same length as dtypes

我声明了要在tf.train.shuffle_batch 函数中使用的形状，但我仍然有形状错误！

知道如何解决这个问题吗？

Answer 1

您的问题都来自

• enqueue_many=True 参数，
• shapes 参数的形状，其中没有 label 维度。

所以我会尝试使用enqueue_many=False 和shapes=[[h, w, c], []])。

确实，如果您查看shuffle_batch 文档：

如果enqueue_many 是False，则假定tensors 代表一个 单例。将输出形状为[x, y, z] 的输入张量 作为形状为[batch_size, x, y, z] 的张量。

如果enqueue_many 是True，则假定tensors 代表一个 一批示例，其中第一个维度按示例索引， 并且tensors 的所有成员在 第一个维度。如果输入张量的形状为[*, x, y, z]，则 输出的形状为[batch_size, x, y, z]。

但在您的代码中，您似乎只将一个文件出列： img_file, label = read_my_png_files(filename_queue) 并将其直接传递给 shuffle_batch 函数： img_batch,label_batch = tf.train.shuffle_batch([img_file,label], ...) 因此缺少* 维度，TensorFlow 期望[img_file,label] 的第一个维度是示例数。

还要记住enqueue_many 和dequeue_many 是独立的；即

• *：你排入队列的示例数，与
无关
• batch_size：从队列中拉出的新批量大小。