如何在tensorflow中传递参数

Question

我想使用 tensorflow 训练一个网络，我选择“Inception_resnet_v2”作为网络（来自here），这是我的训练代码，

def train(train_dir, annotations, max_step, checkpoint_dir='./checkpoint2/'):
# train the model
features = tf.placeholder("float32", shape=[None, IMAGE_SIZE, IMAGE_SIZE, IMAGE_CHANNEL], name="features")
labels = tf.placeholder("float32", [None], name="labels")
one_hot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=80)
keep_prob = tf.placeholder("float32")
isTraining = tf.placeholder("bool")
#train_step, cross_entropy, logits, keep_prob = network.inference(features, one_hot_labels)
logits, _=inception_resnet_v2.inception_resnet_v2(features,80,isTraining,keep_prob)
# calculate loss
cross_entropy = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(labels=one_hot_labels, logits=logits))



train_step = tf.train.AdamOptimizer(LEARNINGRATE).minimize(cross_entropy)



correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_labels, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))


image_list, label_list = scene_input2.get_files(train_dir, annotations)
image_batch, label_batch = scene_input2.get_batch(image_list, label_list, IMAGE_SIZE, IMAGE_SIZE, BATCH_SIZE)

with tf.Session() as sess:
    saver = tf.train.Saver()
    ckpt = tf.train.get_checkpoint_state(checkpoint_dir)
    if ckpt and ckpt.model_checkpoint_path:
        print('Restore the model from checkpoint %s' % ckpt.model_checkpoint_path)
        # Restores from checkpoint
        saver.restore(sess, ckpt.model_checkpoint_path)
        start_step = int(ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1])
    else:
        sess.run(tf.global_variables_initializer())
        start_step = 0
        print('start training from new state')
    logger = scene_input.train_log(LOGNAME)

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)

    try:
        #        Check if stop was requested.
        step=start_step
        while not coord.should_stop() and step<start_step + max_step:
            start_time = time.time()
            x, y = sess.run([image_batch, label_batch])
            #y = tf.one_hot(indices=tf.cast(y, tf.int32), depth=80)
            #y = sess.run(y)
            sess.run(train_step, feed_dict={features: x, labels: y, isTraining: True, keep_prob: 0.5})
            if step % 50 == 0:
                train_accuracy = sess.run(accuracy, feed_dict={features: x, labels: y, isTraining: False, keep_prob: 1})
                train_loss = sess.run(cross_entropy, feed_dict={features: x, labels: y, isTraining:False, keep_prob: 1})
                duration = time.time() - start_time
                logger.info("step %d: training accuracy %g, loss is %g (%0.3f sec)" % (step, train_accuracy, train_loss, duration))
            if step % 1000 == 1:
                saver.save(sess, CHECKFILE, global_step=step)
                print('writing checkpoint at step %s' % step)
            step=step+1


    except tf.errors.OutOfRangeError:
        print('done!')
    finally:
        #Request that the threads stop.After this is called, calls to should_stop() will return True.
        coord.request_stop()
    coord.join(threads)

但是当我训练网络时，我遇到了一个错误：

    Traceback (most recent call last):
  File "scene2.py", line 245, in <module>
    train(FLAGS.train_dir, FLAGS. annotations, FLAGS.max_step)
  File "scene2.py", line 82, in train
    logits, _=inception_resnet_v2.inception_resnet_v2(features,80,isTraining,keep_prob)
  File "/home/vision/inception_resnet_v2.py", line 357, in inception_resnet_v2
    scope='Dropout')
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
    return func(*args, **current_args)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1216, in dropout
    _scope=sc)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/layers/core.py", line 247, in __init__
    self.rate = min(1., max(0., rate))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 564, in __bool__
    raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "
TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
vision@Hjl:~/$ CUDA_VISIBLE_DEVICES=0 python3 scene2.py --mode train
Traceback (most recent call last):
  File "scene2.py", line 245, in <module>
    train(FLAGS.train_dir, FLAGS. annotations, FLAGS.max_step)
  File "scene2.py", line 82, in train
    logits, _=inception_resnet_v2.inception_resnet_v2(features,80,isTraining,keep_prob)
  File "/home/vision/inception_resnet_v2.py", line 357, in inception_resnet_v2
    scope='Dropout')
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
    return func(*args, **current_args)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1216, in dropout
    _scope=sc)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/layers/core.py", line 247, in __init__
    self.rate = min(1., max(0., rate))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 564, in __bool__
    raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "
TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

当我将 keep_prob 和 keep_prob 传递给 inception_resnet_v2.inception_resnet_v2(features,80,isTraining,keep_prob) 时，发生了错误。我该如何解决这个问题？

当我训练网络时，我想设置keep_prob = 0.5，isTraining = True，但同时，每50步，我想观察模型的train_accuracy和train_loss，所以我应该设置keep_prob = 1.0，isTraining = 错，我说的对吗？如何实现？

Answer 1

如果您的最终目标是同时执行训练和评估，并且您正在使用 tf-slim 库提供的神经网络实现，那么遵循the methodology prescribed by tf-slim co-author Nathan Silberman 可能是最简单的。

简而言之，训练和评估是由两个独立的进程执行的，评估进程指向一个检查点目录，（无限地）等待训练进程将新的检查点写入该目录，然后自动执行评估在新写入的检查点上并将摘要写入指定的 eval 输出目录。

要开始使用，您应该查看the TensorFlow-Slim image classification model library 中提供的 train_image_classifier.py 和 eval_image_classifier.py 脚本。

在 eval_image_classifier.py 中，您需要替换代码：

if tf.gfile.IsDirectory(FLAGS.checkpoint_path):
  checkpoint_path = tf.train.latest_checkpoint(FLAGS.checkpoint_path)
else:
  checkpoint_path = FLAGS.checkpoint_path

tf.logging.info('Evaluating %s' % checkpoint_path)

slim.evaluation.evaluate_once(
    master=FLAGS.master,
    checkpoint_path=checkpoint_path,
    logdir=FLAGS.eval_dir,
    num_evals=num_batches,
    eval_op=list(names_to_updates.values()),
    variables_to_restore=variables_to_restore)

用代码：

tf.logging.info('Evaluating %s' % FLAGS.checkpoint_path)

slim.evaluation.evaluation_loop(
    master=FLAGS.master,
    checkpoint_dir=FLAGS.checkpoint_path,
    logdir=FLAGS.eval_dir,
    num_evals=num_batches,
    eval_op=list(names_to_updates.values()),
    variables_to_restore=variables_to_restore)

如果您希望两个进程都使用您的 GPU 而不会遇到 OOM 错误，您可以通过创建 ConfigProto 对象并将其作为 session_config 参数的参数传递给每个进程分配一小部分 GPU 内存slim.learning.train() 或 slim.evaluation.evaluation_loop()。请参阅this tensorflow.org article 的“允许 GPU 内存增长”部分以供参考。

关于is_training的参数化，您会注意到train和eval脚本分别将True和False作为参数传递给nets_factory.get_netowrk_fn()的is_training参数。

关于keep_prob的参数化，nets_factory没有暴露slim nets的dropout_keep_prob参数。相反，slim.dropout() 接受 is_training 作为参数，并将构成 dropout 的计算替换为恒等函数。换句话说，tf-slim 非常棒，当你将is_training=False 传递给nets_factory.get_netowrk_fn() 时，它会自动“禁用” dropout，就像 eval_image_classifier.py 中的情况一样。

如果您想将dropout_keep_prob 直接暴露给 train_image_classifier.py（例如，出于超参数调整的目的），您将不得不摆弄nets_factory.get_network_fn() 的实现。

Answer 2

如果您使用this 方法，那么它需要一个python boolean 和float 值而不是tensor。所以你需要传递类似的值，

keep_prob = 0.5
isTraining = True

而不是

keep_prob = tf.placeholder("float32")
isTraining = tf.placeholder("bool")

更新

但是如果你需要在训练时间喂他们，我认为最简单的方法是在这个line 编辑inception_resnet_v2 方法参数，如下所示（删除默认参数值），

def inception_resnet_v2(inputs, num_classes=1001, is_training,
                        dropout_keep_prob,
                        reuse=None,
                        scope='InceptionResnetV2',
                        create_aux_logits=True,
                        activation_fn=tf.nn.relu):

那么您将能够传递您的keep_prob 和isTraining。希望对你有帮助