张量流。从 BasicRNNCell 切换到 LSTMCell答案

【问题标题】：Tensorflow. Switching from BasicRNNCell to LSTMCell张量流。从 BasicRNNCell 切换到 LSTMCell
【发布时间】：2017-07-11 02:07:32
【问题描述】：

我已经用 BasicRNN 构建了一个 RNN，现在我想使用 LSTMCell，但这篇文章似乎并不简单。我应该改变什么？

首先我定义了所有的占位符和变量：

X_placeholder = tf.placeholder(tf.float32, [batch_size, truncated_backprop_length, embedding_size])
Y_placeholder = tf.placeholder(tf.int32, [batch_size, truncated_backprop_length])

init_state = tf.placeholder(tf.float32, [batch_size, state_size])

W = tf.Variable(np.random.rand(state_size, num_classes),dtype=tf.float32)
b = tf.Variable(np.zeros((batch_size, num_classes)), dtype=tf.float32)

W2 = tf.Variable(np.random.rand(state_size, num_classes),dtype=tf.float32)
b2 = tf.Variable(np.zeros((batch_size, num_classes)), dtype=tf.float32)

然后我解开标签：

labels_series = tf.transpose(batchY_placeholder)
labels_series = tf.unstack(batchY_placeholder, axis=1)
inputs_series = X_placeholder

然后我定义我的 RNN：

cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple = False)
states_series, current_state = tf.nn.dynamic_rnn(cell, inputs_series, initial_state = init_state)

我得到的错误是：

InvalidArgumentError                      Traceback (most recent call last)
/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py in _call_cpp_shape_fn_impl(op, input_tensors_needed, input_tensors_as_shapes_needed, debug_python_shape_fn, require_shape_fn)
    669           node_def_str, input_shapes, input_tensors, input_tensors_as_shapes,

--> 670           status)
    671   except errors.InvalidArgumentError as err:

/home/deepnlp2017/anaconda3/lib/python3.5/contextlib.py in __exit__(self, type, value, traceback)
     65             try:
---> 66                 next(self.gen)
     67             except StopIteration:

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
    468           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 469           pywrap_tensorflow.TF_GetCode(status))
    470   finally:

InvalidArgumentError: Dimensions must be equal, but are 50 and 100 for 'rnn/while/basic_lstm_cell/mul' (op: 'Mul') with input shapes: [32,50], [32,100].

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-19-2ac617f4dde4> in <module>()
      4 #cell = tf.contrib.rnn.BasicRNNCell(state_size)
      5 cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple = False)
----> 6 states_series, current_state = tf.nn.dynamic_rnn(cell, inputs_series, initial_state = init_state)

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py in dynamic_rnn(cell, inputs, sequence_length, initial_state, dtype, parallel_iterations, swap_memory, time_major, scope)
    543         swap_memory=swap_memory,
    544         sequence_length=sequence_length,
--> 545         dtype=dtype)
    546 
    547     # Outputs of _dynamic_rnn_loop are always shaped [time, batch, depth].

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py in _dynamic_rnn_loop(cell, inputs, initial_state, parallel_iterations, swap_memory, sequence_length, dtype)
    710       loop_vars=(time, output_ta, state),
    711       parallel_iterations=parallel_iterations,
--> 712       swap_memory=swap_memory)
    713 
    714   # Unpack final output if not using output tuples.

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name)
   2624     context = WhileContext(parallel_iterations, back_prop, swap_memory, name)
   2625     ops.add_to_collection(ops.GraphKeys.WHILE_CONTEXT, context)
-> 2626     result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
   2627     return result
   2628 

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py in BuildLoop(self, pred, body, loop_vars, shape_invariants)
   2457       self.Enter()
   2458       original_body_result, exit_vars = self._BuildLoop(
-> 2459           pred, body, original_loop_vars, loop_vars, shape_invariants)
   2460     finally:
   2461       self.Exit()

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py in _BuildLoop(self, pred, body, original_loop_vars, loop_vars, shape_invariants)
   2407         structure=original_loop_vars,
   2408         flat_sequence=vars_for_body_with_tensor_arrays)
-> 2409     body_result = body(*packed_vars_for_body)
   2410     if not nest.is_sequence(body_result):
   2411       body_result = [body_result]

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py in _time_step(time, output_ta_t, state)
    695           skip_conditionals=True)
    696     else:
--> 697       (output, new_state) = call_cell()
    698 
    699     # Pack state if using state tuples

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py in <lambda>()
    681 
    682     input_t = nest.pack_sequence_as(structure=inputs, flat_sequence=input_t)
--> 683     call_cell = lambda: cell(input_t, state)
    684 
    685     if sequence_length is not None:

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in __call__(self, inputs, state, scope)
    182       i, j, f, o = array_ops.split(value=concat, num_or_size_splits=4, axis=1)
    183 
--> 184       new_c = (c * sigmoid(f + self._forget_bias) + sigmoid(i) *
    185                self._activation(j))
    186       new_h = self._activation(new_c) * sigmoid(o)

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py in binary_op_wrapper(x, y)
    882       if not isinstance(y, sparse_tensor.SparseTensor):
    883         y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
--> 884       return func(x, y, name=name)
    885 
    886   def binary_op_wrapper_sparse(sp_x, y):

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py in _mul_dispatch(x, y, name)
   1103   is_tensor_y = isinstance(y, ops.Tensor)
   1104   if is_tensor_y:
-> 1105     return gen_math_ops._mul(x, y, name=name)
   1106   else:
   1107     assert isinstance(y, sparse_tensor.SparseTensor)  # Case: Dense * Sparse.

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py in _mul(x, y, name)
   1623     A `Tensor`. Has the same type as `x`.
   1624   """
-> 1625   result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
   1626   return result
   1627 

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py in apply_op(self, op_type_name, name, **keywords)
    761         op = g.create_op(op_type_name, inputs, output_types, name=scope,
    762                          input_types=input_types, attrs=attr_protos,
--> 763                          op_def=op_def)
    764         if output_structure:
    765           outputs = op.outputs

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in create_op(self, op_type, inputs, dtypes, input_types, name, attrs, op_def, compute_shapes, compute_device)
   2395                     original_op=self._default_original_op, op_def=op_def)
   2396     if compute_shapes:
-> 2397       set_shapes_for_outputs(ret)
   2398     self._add_op(ret)
   2399     self._record_op_seen_by_control_dependencies(ret)

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in set_shapes_for_outputs(op)
   1755       shape_func = _call_cpp_shape_fn_and_require_op
   1756 
-> 1757   shapes = shape_func(op)
   1758   if shapes is None:
   1759     raise RuntimeError(

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in call_with_requiring(op)
   1705 
   1706   def call_with_requiring(op):
-> 1707     return call_cpp_shape_fn(op, require_shape_fn=True)
   1708 
   1709   _call_cpp_shape_fn_and_require_op = call_with_requiring

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py in call_cpp_shape_fn(op, input_tensors_needed, input_tensors_as_shapes_needed, debug_python_shape_fn, require_shape_fn)
    608     res = _call_cpp_shape_fn_impl(op, input_tensors_needed,
    609                                   input_tensors_as_shapes_needed,
--> 610                                   debug_python_shape_fn, require_shape_fn)
    611     if not isinstance(res, dict):
    612       # Handles the case where _call_cpp_shape_fn_impl calls unknown_shape(op).

/home/deepnlp2017/.local/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py in _call_cpp_shape_fn_impl(op, input_tensors_needed, input_tensors_as_shapes_needed, debug_python_shape_fn, require_shape_fn)
    673       missing_shape_fn = True
    674     else:
--> 675       raise ValueError(err.message)
    676 
    677   if missing_shape_fn:

ValueError: Dimensions must be equal, but are 50 and 100 for 'rnn/while/basic_lstm_cell/mul' (op: 'Mul') with input shapes: [32,50], [32,100].

【问题讨论】：

“这段话似乎并不简单。”你面临什么问题？
这是我定义单元格的代码：#cell = tf.contrib.rnn.BasicRNNCell(state_size) cell = tf.contrib.rnn.LSTMCell(state_size) states_series, current_state = tf. nn.dynamic_rnn(cell, inputs_series, initial_state = init_state) 但是当谈到“tf.nn.dynamic_rnn”这一行时，它给了我错误：TypeError: 'Tensor' object is not iterable。
我在下面更新了我的答案
我的 Tensorflow 版本是 1.0.0。我试图将 state_is_tuple 设置为 False，现在我收到以下错误：ValueError: Dimensions must be equal, but are 50 and 100 for 'rnn/while/basic_lstm_cell/mul' (op: 'Mul') with input shapes: [32 ,50]，[32,100]。（为什么要把 state_is_tuple 设置为 false？？？）

标签： machine-learning tensorflow neural-network deep-learning recurrent-neural-network

【解决方案1】：

您应该考虑给出错误跟踪。否则很难（或不可能）提供帮助。

我复现了这个情况，发现问题出在状态解包，即c, h = state行。

尝试将 state_is_tuple 设置为 false 即

cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple=False)

我不确定为什么会这样。您是否正在加载以前的模型？你的 tensorflow 版本是多少？

有关 TensorFlow RNN 单元的更多信息：

我建议您看一下：WildML post，“RNN 细胞、包装器和多层 RNNS”部分。

它指出：

BasicRNNCell – 一个普通的 RNN 单元。

GRUCell – 门控循环单元单元。

BasicLSTMCell – 基于循环神经网络正则化的 LSTM 单元。没有窥视孔连接或细胞夹。

LSTMCell – 一个更复杂的 LSTM 单元，允许可选的窥视孔连接和单元剪辑。

MultiRNNCell – 将多个单元组合成多层单元的包装器。

DropoutWrapper – 一种用于将 dropout 添加到单元的输入和/或输出连接的包装器。

鉴于此，我建议您从BasicRNNCell 切换到BasicLSTMCell。 Basic 这里的意思是“除非你知道自己在做什么，否则使用它”。如果您想尝试 LSTMs 而无需详细说明，那就是要走的路。它可能很简单，只需替换它即可！

如果没有，请分享您的一些代码+错误。

希望对你有帮助

【讨论】：

这是我定义单元格的代码：#cell = tf.contrib.rnn.BasicRNNCell(state_size) cell = tf.contrib.rnn.LSTMCell(state_size) states_series, current_state = tf. nn.dynamic_rnn(cell, inputs_series, initial_state = init_state) 但是当谈到“tf.nn.dynamic_rnn”这一行时，它给了我错误：TypeError: 'Tensor' object is not iterable。
我的回答建议使用BasicLSTMCell。也请把你的代码放在问题中
对不起，我想尝试他们两个，看看我是否遇到了同样的错误......但是我添加了代码（如果您需要更多，请询问）

【解决方案2】：

问题似乎出在init_state 变量上。基本的 RNN 单元只有一个状态变量，而 LSTM 有可见和隐藏状态。指定选项state_is_tuple=False 会将两个状态变量合并为一个，因此是您在 init_state 声明中指定的大小的两倍。为避免这种情况，可以使用 LSTMCell 的内置 zero_state 方法以正确的方式初始化状态，而无需担心大小差异。所以它只是：

init_state = cell.zero_state(batch_size, dtype)

当然必须放在构建单元格的行之后。

【讨论】：