使用输入管道时如何替换 feed_dict？

Question

假设您有一个迄今为止与feed_dict 合作的网络将数据注入图表。每隔几个 epoch，我通过将任一数据集的批次输入到我的图表中来评估训练和测试损失。

现在，出于性能原因，我决定使用输入管道。看看这个虚拟的例子：

import tensorflow as tf
import numpy as np

dataset_size = 200
batch_size= 5
dimension = 4

# create some training dataset
dataset = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_size,dimension)).
    astype(np.float32))

dataset = dataset.batch(batch_size) # take batches

iterator = dataset.make_initializable_iterator()
x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(iterator.initializer)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

        # I want to print the loss from another dataset (test set) here

打印训练数据的损失没有问题，但是我如何为另一个数据集执行此操作？ 当使用feed_dict 时，我只是从所述集合中获取一个批次并为其输入一个值为 x。

Answer 1

您可以为此做几件事。一个简单的选择可能是拥有两个数据集和迭代器，并使用tf.cond 在它们之间切换。然而，更强大的方法是使用直接支持这一点的迭代器。有关各种迭代器类型的描述，请参阅how to create iterators 上的指南。例如，使用一个可重新初始化的迭代器，你可以有这样的东西：

import tensorflow as tf
import numpy as np

dataset_size = 200
dataset_test_size = 20
batch_size= 5
dimension = 4

# create some training dataset
dataset = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_size,dimension)).
    astype(np.float32))

dataset = dataset.batch(batch_size) # take batches

# create some test dataset
dataset_test = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_test_size,dimension)).
    astype(np.float32))

dataset_test = dataset_test.batch(batch_size) # take batches

iterator = tf.data.Iterator.from_structure(dataset.output_types,
                                           dataset.output_shapes)

dataset_init_op = iterator.make_initializer(dataset)
dataset_test_init_op = iterator.make_initializer(dataset_test)

x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(dataset_init_op)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

    # print test loss
    sess.run(dataset_test_init_op)
    for i in range(dataset_test_size//batch_size):
        loss1 = sess.run(loss)
        print('test step {:d}.  batch loss {:f}.'.format(i,loss1))

您可以使用可馈送迭代器执行类似的操作，具体取决于您认为更方便的方式，我想即使使用可初始化的迭代器，例如制作一个布尔数据集，然后您使用 tf.cond 映射到一些数据，尽管那这样做不是很自然的方式。

---

编辑：

以下是使用可初始化迭代器的方法，实际上比我最初想的更简洁，所以也许你更喜欢这个：

import tensorflow as tf
import numpy as np

dataset_size = 200
dataset_test_size = 20
batch_size= 5
dimension = 4

# create data
data = tf.constant(np.random.normal(2.0,size=(dataset_size,dimension)), tf.float32)
data_test = tf.constant(np.random.normal(2.0,size=(dataset_test_size,dimension)), tf.float32)
# choose data
testing = tf.placeholder_with_default(False, ())
current_data = tf.cond(testing, lambda: data_test, lambda: data)
# create dataset
dataset = tf.data.Dataset.from_tensor_slices(current_data)
dataset = dataset.batch(batch_size)
# create iterator
iterator = dataset.make_initializable_iterator()

x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(iterator.initializer)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

    # print test loss
    sess.run(iterator.initializer, feed_dict={testing: True})
    for i in range(dataset_test_size//batch_size):
        loss1 = sess.run(loss)
        print('test step {:d}.  batch loss {:f}.'.format(i,loss1))