Tensorflow 1.1 MultiRNNCell 形状错误（Init_State 相关）

Question

更新：我坚信该错误与创建并作为参数输入 tf.nn.dynamic_rnn(...) 的 init_state 相关。那么问题就变成了，堆叠 RNN 的初始状态的正确形状或构造方法是什么？

我正在尝试让 MultiRNNCell 定义在 TensorFlow 1.1 中工作。

使用辅助函数定义 GRU 单元的图形定义如下。基本思想是将占位符 x 定义为一长串数字数据样本。这些数据将通过整形分成等长的帧，每个时间步都会呈现一帧。然后，我想通过两个（目前）GRU 单元的堆栈来处理这个问题。

def gru_cell(state_size):
     cell = tf.contrib.rnn.GRUCell(state_size)
     return cell

graph = tf.Graph()
with graph.as_default():

     x = tf.placeholder(tf.float32, [batch_size, num_samples], name="Input_Placeholder")
     y = tf.placeholder(tf.int32, [batch_size, num_frames], name="Labels_Placeholder")

     init_state = tf.zeros([batch_size, state_size], name="Initial_State_Placeholder")

     rnn_inputs = tf.reshape(x, (batch_size, num_frames, frame_length))
     cell = tf.contrib.rnn.MultiRNNCell([gru_cell(state_size) for _ in range(2)], state_is_tuple=False)
     rnn_outputs, final_state = tf.nn.dynamic_rnn(cell, rnn_inputs, initial_state=init_state) 

图形定义从那里继续使用损失函数、优化器等。但这就是它因以下冗长错误而崩溃的地方。

batch_size 为 10，frame_length 和 state_size 均为 80，在错误的最后部分将变得相关。

ValueError                                Traceback (most recent call last)
<ipython-input-30-4c48b596e055> in <module>()
     14     print(rnn_inputs)
     15     cell = tf.contrib.rnn.MultiRNNCell([gru_cell(state_size) for _ in range(2)], state_is_tuple=False)
---> 16     rnn_outputs, final_state = tf.nn.dynamic_rnn(cell, rnn_inputs, initial_state=init_state)
     17 
     18     with tf.variable_scope('softmax'):

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/rnn.pyc in dynamic_rnn(cell, inputs, sequence_length, initial_state, dtype, parallel_iterations, swap_memory, time_major, scope)
    551         swap_memory=swap_memory,
    552         sequence_length=sequence_length,
--> 553         dtype=dtype)
    554 
    555     # Outputs of _dynamic_rnn_loop are always shaped [time, batch, depth].

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/rnn.pyc in _dynamic_rnn_loop(cell, inputs, initial_state, parallel_iterations, swap_memory, sequence_length, dtype)
    718       loop_vars=(time, output_ta, state),
    719       parallel_iterations=parallel_iterations,
--> 720       swap_memory=swap_memory)
    721 
    722   # Unpack final output if not using output tuples.

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.pyc in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name)
   2621     context = WhileContext(parallel_iterations, back_prop, swap_memory, name)
   2622     ops.add_to_collection(ops.GraphKeys.WHILE_CONTEXT, context)
-> 2623     result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
   2624     return result
   2625 

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.pyc in BuildLoop(self, pred, body, loop_vars, shape_invariants)
   2454       self.Enter()
   2455       original_body_result, exit_vars = self._BuildLoop(
-> 2456           pred, body, original_loop_vars, loop_vars, shape_invariants)
   2457     finally:
   2458       self.Exit()

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.pyc in _BuildLoop(self, pred, body, original_loop_vars, loop_vars, shape_invariants)
   2435     for m_var, n_var in zip(merge_vars, next_vars):
   2436       if isinstance(m_var, ops.Tensor):
-> 2437         _EnforceShapeInvariant(m_var, n_var)
   2438 
   2439     # Exit the loop.

/home/novak/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.pyc in _EnforceShapeInvariant(merge_var, next_var)
    565           "Provide shape invariants using either the `shape_invariants` "
    566           "argument of tf.while_loop or set_shape() on the loop variables."
--> 567           % (merge_var.name, m_shape, n_shape))
    568   else:
    569     if not isinstance(var, (ops.IndexedSlices, sparse_tensor.SparseTensor)):

ValueError: The shape for rnn/while/Merge_2:0 is not an invariant for the loop. It enters the loop with shape (10, 80), but has shape (10, 160) after one iteration. Provide shape invariants using either the `shape_invariants` argument of tf.while_loop or set_shape() on the loop variables.

这几乎看起来网络从 80 的 2 堆栈开始，然后以某种方式转换为 160 的 1 堆栈。解决这个问题有什么帮助吗？我对 MultiRNNCell 的使用有误解吗？

Answer 1

根据上面 Allen Lavoie 的评论，更正的代码是：

def gru_cell(state_size):
     cell = tf.contrib.rnn.GRUCell(state_size)
     return cell

num_layers = 2  # <---------
graph = tf.Graph()
with graph.as_default():

     x = tf.placeholder(tf.float32, [batch_size, num_samples], name="Input_Placeholder")
     y = tf.placeholder(tf.int32, [batch_size, num_frames], name="Labels_Placeholder")

     init_state = tf.zeros([batch_size, num_layer * state_size], name="Initial_State_Placeholder") # <---------

     rnn_inputs = tf.reshape(x, (batch_size, num_frames, frame_length))
     cell = tf.contrib.rnn.MultiRNNCell([gru_cell(state_size) for _ in range(num_layer)], state_is_tuple=False) # <---------
     rnn_outputs, final_state = tf.nn.dynamic_rnn(cell, rnn_inputs, initial_state=init_state) 

请注意上面的三个更改。 另请注意，这些更改必须在 init_state 流动的所有地方产生涟漪，尤其是当您将它们输入到 feed_dict 时。