如何将先前的时间戳预测作为下一个时间戳的附加输入？

Question

可能有人问过这个问题，但我很困惑。

我正在尝试应用一种 RNN 类型，例如用于时间序列预测的 LSTM。我有输入，y（股票收益）。对于每个时间戳，我想得到预测。 Q1 - 我选择 seq2seq 方法是否正确？

我还想使用来自先前时间戳的预测（使用一些常数初始化初始值）作为平方残差形式的附加（仍然使用我现有的输入）输入，即使用 eps_{t-1} = (y_{t-1} - y^_{t-1})^2 作为 t 处的附加输入（以及之前的输入）。

那么，我如何在 tensorflow 或 pytorch 中做到这一点？

我试图在附图中描述我想要的内容。 The graph

附言抱歉，问题表述不当

Answer 1

假设您输入的维度为 (32,10,1)，batch_size 为 32，时间步长为 10，维度为 1。您的目标（股票回报）也一样。此代码使用tf.scan 函数，该函数在实现自定义循环网络时很有用（它将迭代时间步长）。如您所愿，仍然可以在某处使用 t-1 的残差。

ps：它是从零开始的一个非常基本的 lstm 实现，没有任何偏差或输出激活。

import tensorflow as tf
import numpy as np
tf.reset_default_graph()

BS = 32
TS = 10
inputs_dim = 1
target_dim = 1

inputs = tf.placeholder(shape=[BS, TS, inputs_dim], dtype=tf.float32)
stock_returns = tf.placeholder(shape=[BS, TS, target_dim], dtype=tf.float32)

state_size = 16

# initial hidden state
init_state = tf.placeholder(shape=[2, BS, state_size], 
                        dtype=tf.float32, name='initial_state')
# initializer
xav_init = tf.contrib.layers.xavier_initializer
# params
W = tf.get_variable('W', shape=[4, state_size, state_size], 
                initializer=xav_init())
U = tf.get_variable('U', shape=[4, inputs_dim, state_size], 
                initializer=xav_init())
W_out = tf.get_variable('W_out', shape=[state_size, target_dim], 
                   initializer=xav_init())

#the function to feed tf.scan with
def step(prev, inputs_):

   #unpack all inputs and previous outputs
   st_1, ct_1 = prev[0][0], prev[0][1]
   x = inputs_[0]
   target = inputs_[1]

   #get previous squared residual
   eps = prev[1]


   """
   here do whatever you want with eps_t-1
   like x += eps if x if of the same dimension
   or include it somewhere in your graph
   """


   # lstm gates (add bias if needed)
   #
   #  input gate
   i = tf.sigmoid(tf.matmul(x,U[0]) + tf.matmul(st_1,W[0]))
   #  forget gate
   f = tf.sigmoid(tf.matmul(x,U[1]) + tf.matmul(st_1,W[1]))
   #  output gate
   o = tf.sigmoid(tf.matmul(x,U[2]) + tf.matmul(st_1,W[2]))
   #  gate weights
   g = tf.tanh(tf.matmul(x,U[3]) + tf.matmul(st_1,W[3]))

   ct = ct_1*f + g*i
   st = tf.tanh(ct)*o

   """
   make prediction, compute residual in t
   and pass it to t+1
   Normaly, we would compute prediction outside the scan function, 
   but as we need it here, we could just keep it and return it back
   as an output of the scan function 
   """

   prediction_t = tf.matmul(st, W_out) # + bias
   eps = (target - prediction_t)**2 

   return [tf.stack((st, ct), axis=0), eps, prediction_t] 

states, eps, preds = tf.scan(step, [tf.transpose(inputs, [1,0,2]), 
       tf.transpose(stock_returns, [1,0,2])], initializer=[init_state, 
       tf.zeros((32,1), dtype=tf.float32), 
       tf.zeros((32,1),dtype=tf.float32)])


with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())
  out = sess.run(preds, feed_dict= 
               {inputs:np.random.rand(BS,TS,inputs_dim),                                      
                stock_returns:np.random.rand(BS,TS,target_dim),                                    
                init_state:np.zeros((2,BS,state_size))})
  out = tf.transpose(out,[1,0,2])
  print(out)

然后输出：

Tensor("transpose_2:0", shape=(32, 10, 1), dtype=float32)

来自here的基本代码