【问题标题】：LSTM X Values are shifted on the prediction?LSTM X 值在预测上发生了变化？
【发布时间】：2019-08-23 23:02:13
【问题描述】：

介绍和问题：

我对 Keras 和深度学习有些陌生，我正在尝试使用 LSTM 预测特斯拉的股票价格。我绝对是机器学习/深度学习的初学者，所以我希望有更多知识和经验的人可以帮助指导我朝着正确的方向前进。

我的网络在 y 值预测上表现良好。但似乎 x 值向原点左移太远了。似乎如果我将数据向右移动，预测实际上会非常好。下面是预测图的图片：

我很确定错误来自于我创建 X_test 值数组时。以下是将以下所有代码组织成部分：

数据集：

我的火车数据是特斯拉从 2014 年到 2018 年的 4 年收盘价。我要预测的数据是 2019 年的收盘价。

# get 2014-2018 data to train our model
start = datetime.datetime(2014,1,1)
end = datetime.datetime(2018,12,30)
df = web.DataReader("TSLA", 'yahoo', start, end) 

# get 2019 data to test our model on 
start = datetime.datetime(2019,1,1)
end = datetime.date.today()
test_df = web.DataReader("TSLA", 'yahoo', start, end) 

# sort by date
df = df.sort_values('Date')
test_df = test_df.sort_values('Date')

# fix the date 
df.reset_index(inplace=True)
df.set_index("Date", inplace=True)
test_df.reset_index(inplace=True)
test_df.set_index("Date", inplace=True)

df.tail()

                  High         Low        Open       Close   Volume  
Date                                                                  
2014-01-02  152.479996  146.550003  149.800003  150.100006  6188400   
2014-01-03  152.190002  148.600006  150.000000  149.559998  4695000   
2014-01-06  150.399994  145.240005  150.000000  147.000000  5361100   
2014-01-07  150.399994  145.250000  147.619995  149.360001  5034100   
2014-01-08  153.699997  148.759995  148.850006  151.279999  6163200   
...                ...         ...         ...         ...      ...   
2018-12-24  314.500000  295.200012  313.500000  295.390015  5559900   
2018-12-26  326.970001  294.089996  300.000000  326.089996  8163100   
2018-12-27  322.170013  301.500000  319.839996  316.130005  8575100   
2018-12-28  336.239990  318.410004  323.100006  333.869995  9939000   
2018-12-31  339.209991  325.260010  337.790009  332.799988  6302300

创建火车数据：

# create train set of adj close prices data:
train_data = df.loc[:,'Adj Close'].as_matrix()
print(train_data.shape) # 1258 


# Apply normalization before feeding to LSTM using sklearn:
from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
train_data = train_data.reshape(-1,1)

scaler.fit(train_data)
train_data = scaler.transform(train_data)


X_train = []
y_train = []
# loop through the data in batches of 36 to create our time steps
print(train_data.shape)
for i in range(36, len(train_data)):
    # append 
    X_train.append(train_data[i-36:i, 0])
    y_train.append(train_data[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)
print(len(X_train))

X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))

for循环前train_data的len为1258，for循环后为1222。

创建我们的模型：

# Build the model 
model = Sequential()

model.add(LSTM(units = 100, return_sequences = True, input_shape = (X_train.shape[1], 1)))
model.add(Dropout(0.2))

model.add(LSTM(units = 100))
model.add(Dropout(0.2))

# Output layer
model.add(Dense(units = 1))

# Compiling the RNN
model.compile(optimizer = 'adam', loss = 'mean_squared_error')

拟合模型

history = model.fit(X_train, y_train, epochs = 20, batch_size = 10, validation_split=.30)

Train on 855 samples, validate on 367 samples
Epoch 1/20
855/855 [==============================] - 14s 17ms/step - loss: 0.0099 - val_loss: 0.0061
Epoch 2/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0035 - val_loss: 0.0059
Epoch 3/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0036 - val_loss: 0.0057
Epoch 4/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0029 - val_loss: 0.0042
Epoch 5/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0024 - val_loss: 0.0092
Epoch 6/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0025 - val_loss: 0.0039
Epoch 7/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0024 - val_loss: 0.0047
Epoch 8/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0027 - val_loss: 0.0034
Epoch 9/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0023 - val_loss: 0.0035
Epoch 10/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0020 - val_loss: 0.0029
Epoch 11/20
855/855 [==============================] - 7s 8ms/step - loss: 0.0018 - val_loss: 0.0033
Epoch 12/20
855/855 [==============================] - 6s 8ms/step - loss: 0.0021 - val_loss: 0.0027
Epoch 13/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0018 - val_loss: 0.0025
Epoch 14/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0019 - val_loss: 0.0038
Epoch 15/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0019 - val_loss: 0.0037
Epoch 16/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0018 - val_loss: 0.0023
Epoch 17/20
855/855 [==============================] - 7s 8ms/step - loss: 0.0017 - val_loss: 0.0025
Epoch 18/20
855/855 [==============================] - 8s 9ms/step - loss: 0.0015 - val_loss: 0.0021
Epoch 19/20
855/855 [==============================] - 8s 9ms/step - loss: 0.0015 - val_loss: 0.0025
Epoch 20/20
855/855 [==============================] - 6s 7ms/step - loss: 0.0016 - val_loss: 0.0023

模型损失图：

创建测试数据（这里可能是BUG）：

# Predict with the model and plot predictions  
inputs = test_df['Adj Close'].values
inputs = inputs.reshape(-1,1)
inputs = scaler.transform(inputs)

print(len(inputs)) # 159 

X_test = []
y_test = []
for i in range(36, len(inputs)):
    X_test.append(inputs[i-36:i, 0])

X_test = np.array(X_test)
print(len(X_test)) #123
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

predictions = model.predict(X_test)
predictions = scaler.inverse_transform(predictions)
#print(predictions)

有趣的是len(inputs) 在进入 for 循环之前是 = 159，而从 for 循环出来 len(X_test) 是 = 123。我正在使用这个 for 循环来创建一批 36 天的库存价格，因此我们的 LSTM 可以在做出预测之前考虑 36 天的价格。但它似乎切断了最后 36 个值？

【问题讨论】：

标签： python keras deep-learning time-series lstm

【解决方案1】：

解决办法：

如果有人从谷歌找到这个我想通了。我将训练和测试数据创建代码更改为：

'''Function to create a dataset to feed into an LSTM'''
def create_dataset(dataset, look_back):
    dataX, dataY = [], []
    for i in range(len(dataset)-look_back):
        a = dataset[i:(i + look_back), 0]
        dataX.append(a)
        dataY.append(dataset[i + look_back, 0])
    return np.array(dataX), np.array(dataY)

# Create the data to train our model on:
time_steps = 36
X_train, y_train = create_dataset(train_data, time_steps)

# reshape it [samples, time steps, features]
X_train = np.reshape(X_train, (X_train.shape[0], 36, 1))

print(X_train.shape) # 1222, 36, 1

测试数据：

# Get the stock prices for 2019 to have our model make the predictions
test_data = test_df['Adj Close'].values
test_data = test_data.reshape(-1,1)
test_data = scaler.transform(test_data)

# Create the data to test our model on:
time_steps = 36
X_test, y_test = create_dataset(test_data, time_steps)

# store the original vals for plotting the predictions 
y_test = y_test.reshape(-1,1)
org_y = scaler.inverse_transform(y_test)

# reshape it [samples, time steps, features]
X_test = np.reshape(X_test, (X_test.shape[0], 36, 1))

# Predict the prices with the model
predicted_y = model.predict(X_test)
predicted_y = scaler.inverse_transform(predicted_y)

新预测：

我将其更改为绘制存储在 org_y 中的原始 y val，然后绘制我们预测的 y val

plt.plot(org_y, color = 'red', label = 'Real Tesla Stock Price')
plt.plot(predicted_y, color = 'blue', label = 'Predicted Tesla Stock Price')
plt.title('Tesla Stock Price Prediction')
plt.xlabel('Time')
plt.ylabel('Tesla Stock Price')
plt.legend()
plt.show()

【讨论】：