使用“slim.learning.train”恢复用于微调的张量流模型

Question

在张量流中，使用 slim.learning.train (TF 0.11)，我想从检查点恢复模型并继续训练。该模型进行了成功的培训，我想对其进行微调。但是，当我这样做时，TF 崩溃并出现错误 Init operations did not make model ready.

我用以下方式进行培训：

tf.contrib.slim.learning.train(
    train_op,
    train_dir,
    log_every_n_steps=FLAGS.log_every_n_steps,
    graph=g,
    global_step=model.global_step,
    number_of_steps=FLAGS.number_of_steps,
    init_fn=model.init_fn,
    saver=model.saver,
    session_config=session_config)

我尝试了 3 种选择：

#1

关注this doc

model.init_fn = None

#2

with g.as_default():
    model_path = tf.train.latest_checkpoint(train_dir)
    if model_path:
        def restore_fn(sess):
            tf.logging.info(
                "Restoring SA&T variables from checkpoint file %s",
                restore_fn.model_path)
            model.saver.restore(sess, restore_fn.model_path)
        restore_fn.model_path = model_path
        model.init_fn = restore_fn
    else:
        model.init_fn = None

#3

with g.as_default():
    model_path = tf.train.latest_checkpoint(train_dir)
    if model_path:
        variables_to_restore = tf.contrib.slim.get_variables_to_restore()
        model.init_fn = tensorflow.contrib.framework.assign_from_checkpoint_fn(
            model_path, variables_to_restore)
    else:
        model.init_fn = None

Answer 1

问题已解决。发生这种情况是因为保护程序 (tf.train.Saver) 是在模型构建后直接定义的。

相反，按照训练操作定义定义它，解决了这个问题。