为什么手动构建 timestep-unfold LSTM 与使用 static_rnn 有不同的输出？答案

【问题标题】：Why build timestep-unfold LSTM manually has different outputs from using static_rnn?为什么手动构建 timestep-unfold LSTM 与使用 static_rnn 有不同的输出？
【发布时间】：2023-11-27 07:40:01
【问题描述】：

这是我手动构建 LSTM 的代码：

import tensorflow as tf
import numpy as np
batch_size = 1
hidden_size = 4
num_steps = 3
input_dim = 5
np.random.seed(123)
input = np.ones([batch_size, num_steps, input_dim], dtype=int)
x = tf.placeholder(dtype=tf.float32, shape=[batch_size, num_steps, input_dim], name='input_x')
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=hidden_size)
initial_state = lstm_cell.zero_state(batch_size, dtype=tf.float32)
outputs = []
with tf.variable_scope('for_loop', initializer= tf.ones_initializer):
    for i in range(num_steps):
        if i > 0:
            tf.get_variable_scope().reuse_variables()
        output = lstm_cell(x[:, i, :], initial_state)
        outputs.append(output)
with tf.Session() as sess:
    init_op = tf.initialize_all_variables()
    sess.run(init_op)
    result = sess.run(outputs, feed_dict={x: input})
    print(result)

输出：

[(array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32), LSTMStateTuple(c=array([[0.99321693, 0.99321693, 0.99321693, 0.99321693]], dtype=float32), h=array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32))), 
(array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32), LSTMStateTuple(c=array([[0.99321693, 0.99321693, 0.99321693, 0.99321693]], dtype=float32), h=array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32))), 
(array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32), LSTMStateTuple(c=array([[0.99321693, 0.99321693, 0.99321693, 0.99321693]], dtype=float32), h=array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32)))]

虽然这是使用 static_rnn 的代码：

import tensorflow as tf
import numpy as np
batch_size = 1
hidden_size = 4
num_steps = 3
input_dim = 5
np.random.seed(123)
input = np.ones([batch_size, num_steps, input_dim], dtype=int)
x = tf.placeholder(dtype=tf.float32, shape=[batch_size, num_steps, input_dim], name='input_x')
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=hidden_size)
initial_state = lstm_cell.zero_state(batch_size, dtype=tf.float32)
y = tf.unstack(x, axis=1)
with tf.variable_scope('static_rnn', initializer= tf.ones_initializer):
    output, state = tf.nn.static_rnn(lstm_cell, y,  initial_state=initial_state)
with tf.Session() as sess:
    init_op = tf.initialize_all_variables()
    sess.run(init_op)
    result = (sess.run([output, state], feed_dict={x: input}))
    print(result)

输出：

[[array([[0.7536526, 0.7536526, 0.7536526, 0.7536526]], dtype=float32), 
array([[0.9631945, 0.9631945, 0.9631945, 0.9631945]], dtype=float32), 
array([[0.9948382, 0.9948382, 0.9948382, 0.9948382]], dtype=float32)], LSTMStateTuple(c=array([[2.9925175, 2.9925175, 2.9925175, 2.9925175]], dtype=float32), h=array([[0.9948382, 0.9948382, 0.9948382, 0.9948382]], dtype=float32))]

第一个单元格获得完全相同的输出，但是由于第二个单元格，手动构建似乎与其前面和后面的单元格没有任何联系——3个单元格的输出相同。我认为手动代码是错误的，但我找不到如何连接 BasicLSTMCell 。救命！

【问题讨论】：

在你的循环中，你在每个时间步传递initial_state。您需要传递上一个时间步的状态。

标签： tensorflow lstm

【解决方案1】：

感谢@Susmit Agrawal，我将代码更改为：

    for i in range(num_steps):
        if i > 0:
            output = lstm_cell(x[:, i, :], outputs[i-1][1])
        else:
            output = lstm_cell(x[:, i, :], z_state)
        outputs.append(output)

这会产生与 static_rnn 相同的正确输出。

【讨论】：