Tensorflow中间歇振荡器的LSTM时间序列预测

Question

我正在构建一个 LSTM 时间序列预测模型（在 TF v=1.13.1，Keras v=2.2.4 中），该模型将间歇性振荡时域信号作为输入。每次振荡之间的时间是指数分布的（beta=5），振荡具有正态分布的长度（mean length=2sec，variance=1sec），并且每次振荡的频率也是正态分布的（mean frequency=22hz，variance=3hz） .

我几乎阅读了 Daniel Möller 关于 Keras/TF 中的 LSTM 主题的所有精彩 Stackoverflow 帖子。我还为我的模型尝试了不同数量的时间步长、有状态与无状态 LSTM、不同的损失函数（mean absolute error 与 mean squared error）以及不同的网络宽度/深度。该模型几乎总是能捕捉到某些东西在振荡，但在顺序预测步骤中未能重建其间歇性特征和正弦形状。

最小可重现示例：

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# Import Data
data = np.loadtxt('./my_data.csv')

# Reshape data into batches of 500 timesteps - 1 dim per timestep
# For now, we do not split into testing/training sets
# Assume all data is for training

data_pre = data.reshape(-1, 500, 1)[:,:-1,:] # Shift input data backward by 1
data_post = data.reshape(-1, 500, 1)[:,1:,:] # Shift input data forward by 1

# Build LSTM Model for Training:

# Allow flexible number of timesteps per input (shape=(None,1))
inputs = tf.keras.layers.Input(shape=(None,1))

lstm_1 = tf.keras.layers.CuDNNLSTM(units=512, return_sequences=True)(inputs)
lstm_2 = tf.keras.layers.CuDNNLSTM(units=256, return_sequences=True)(lstm_1)

# Activate dense layer with linear activation func for regression
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(units=1, activation='linear'))(lstm_2)

lstm_model = tf.keras.Model(inputs=inputs, outputs=outputs)
lstm_model.compile('adam', loss='mae', metrics=['mae','mse'])
lstm_model.fit(x=data_pre, y = data_post, epochs=100, batch_size=16, shuffle=False) # I have trained up to 500 epochs and while the loss decreases there is no increase in prediction performance.

# Build Stateful LSTM Model for Sample-by-Sample Prediction

# Assume 1 timestep per input of dim=1
inputs = tf.keras.layers.Inputs(shape=(1,1,1))
lstm_1 = tf.keras.layers.CuDNNLSTM(units=512, return_sequences=True, stateful=True)(inputs)
lstm_2 = tf.keras.layers.CuDNNLSTM(units=256, return_sequences=True, stateful=True)(lstm_1)

outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(units=1, activation='linear'))

prediction_model = tf.keras.Model(inputs=inputs, outputs=outputs)

# Copy weights from trained, non-stateful model:
prediction_model.set_weights(lstm_model.get_weights())

#Reset network state

prediction_model.reset_states()

#Initialize model internal state with a single sample from the input data shifted by 1 unit backwards

seed = prediction_model.predict(data_pre[0][0][None, None, :])

# Predict 20secs of data

output_array = np.zeros((10000,1,1)) # Allocate Memory

for i in range(0,10000):
    temp = prediction_model.predict(seed) # Iteratively predict next sample value
    output_array[i] = temp
    seed = temp

原始数据图：

模型输出

Answer 1

您是否考虑过通过每一层提供多个输入？例如：
假设您有变量x，您正在通过您的模型进行输入。重塑数据看起来像这样。

import numpy as np
look_back = 5 ## This is the number of points to include in each iteration
x = np.arange(0,100)
new_array = []
for num in range(0, len(x)-look_back):
   new_array.append(x[num:num+look_back])
new_array = np.array(new_array)

print (np.array(x).shape) ## old array = (100,)
print (new_array.shape) ## new array = (95,5)

如果有足够的历史背景，这可能有助于您的模型了解接下来会发生什么。基于振荡方差，我会说这会让你的网络失去活力，它只是在全面取平均值。