ValueError：变量 cudnn_gru/opaque_kernel/ 的初始化程序来自控制流构造内部，例如循环或条件

Question

系统信息

• 我是否编写了自定义代码（而不是使用 TensorFlow 中提供的股票示例脚本）：是
• 操作系统平台和发行版：Ubuntu 16.04
• TensorFlow 安装自：二进制
• TensorFlow 版本：1.5.0
• Python 版本：3.5.2
• CUDA/cuDNN 版本：9.0/7
• GPU 型号和内存：NVIDIA GTX 1080 8GB x 4
• 复制的确切命令：代码如下

我一直在尝试使用在cudnn_gru 上应用tf.cond 的for 循环在输入上运行多次，并且最小错误可重现代码如下：

import tensorflow as tf
max_para = tf.placeholder(tf.int32)
num_units = 150
inputs = tf.placeholder(tf.float32,shape=[15,8,num_units])
class cudnn_gru:
    def __init__(self):
        self.gru_fw = tf.contrib.cudnn_rnn.CudnnGRU(1, num_units, 
            kernel_initializer=tf.random_normal_initializer(stddev=0.1))
        with tf.variable_scope('CUDNN_GRU', reuse=tf.AUTO_REUSE):
            self.init_fw = tf.get_variable("init_fw",shape=[1, 8, num_units],initializer=
                tf.zeros_initializer())
            self.init_bw = tf.get_variable("init_bw",shape=[1, 8, num_units],initializer=
                tf.zeros_initializer())
    def __call__(self,inputs):
        out_fw, _ = self.gru_fw(inputs, initial_state=(self.init_fw,))

class cudnn_gru2:
    def __init__(self):
        self.gru_fw = tf.contrib.cudnn_rnn.CudnnGRU(1, num_units-1, 
            kernel_initializer=tf.random_normal_initializer(stddev=0.1))
        with tf.variable_scope('CUDNN_GRU', reuse=tf.AUTO_REUSE):
            self.init_fw = tf.get_variable("init_fw",shape=[1, 8, num_units],initializer=
                tf.zeros_initializer())
            self.init_bw = tf.get_variable("init_bw",shape=[1, 8, num_units],initializer=
                tf.zeros_initializer())
    def __call__(self,inputs):
        out_fw, _ = self.gru_fw(inputs, initial_state=(self.init_fw,))

def get_output():
    gru = cudnn_gru()
    out = gru(inputs)
    return tf.constant(1)

def get_output2():
    gru = cudnn_gru2()
    out = gru(inputs)
    return tf.constant(2)

for i in range(3):
    i_ = tf.constant(i)
    out = tf.cond(i_<max_para,get_output,get_output2)

错误堆栈跟踪如下：

Traceback (most recent call last):
  File "temp.py", line 41, in <module>
    out = tf.cond(i_<max_para,get_output,get_output2)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func
    return func(*args, **kwargs)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1894, in cond
    orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1752, in BuildCondBranch
    original_result = fn()
  File "temp.py", line 31, in get_output
    out = gru(inputs)
  File "temp.py", line 15, in __call__
    out_fw, _ = self.gru_fw(inputs, initial_state=(self.init_fw,))
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 636, in __call__
    self.build(input_shapes)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py", line 357, in build
    "opaque_kernel", initializer=opaque_params_t, validate_shape=False)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1262, in get_variable
    constraint=constraint)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1097, in get_variable
    constraint=constraint)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 427, in get_variable
    return custom_getter(**custom_getter_kwargs)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py", line 290, in _update_trainable_weights
    variable = getter(*args, **kwargs)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 404, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 806, in _get_single_variable
    constraint=constraint)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 229, in __init__
    constraint=constraint)
  File "/home/search/snetP/virtual_bhavya/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 342, in _init_from_args
    "initializer." % name)
ValueError: Initializer for variable cudnn_gru/opaque_kernel/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.

如何在模型构建过程中不导致编译错误的情况下实现这一点？

Answer 1

tf.cond 和tf.while 循环通过调用真/假函数或循环条件/主体函数仅一次 工作。这些函数返回一些 OP，或者一组 OP 执行一些功能，例如检查条件并返回 true/false，或者在 while 主体的情况下执行一些操作。

Tensorflow 只需调用您的函数一次即可获得所需操作的定义。 Tensorflow 将在循环中或根据需要处理调用该组操作。

请注意，这意味着依赖关系图是静态的，循环执行时没有任何变化。 TensorFlow 只是简单地调整依赖关系以及依赖关系图中的下一步是什么。

因此，如果您要创建一个变量作为循环的一部分，您将更改依赖关系图的结构 - 这不是循环/条件的设计方式。

您需要在条件或循环函数之外定义变量。您可以从循环函数中访问变量，但不能在那里创建它。

请注意，在 RNN/GRU 中，您只有一组在每个时间步应用的权重。所以你不需要在条件/while函数中调用tf.get_variable，你当然可以预先创建你需要的3个变量。

另外，我认为您将函数定义为类的方式是错误的。由于 tensorflow 只是在整个类过度杀伤并且可能令人困惑的事情时才调用该函数。

这是一个我在一段时间前编写的 tf while 循环的示例，它可以为您提供一个很好的示例。我会尝试简化它以遵循此示例格式（例如删除类）。

Tensorflow: How to implement cumulative maximum?