机器学习模型过拟合

Question

所以我构建了一个 GRU 模型，并在同一模型上比较了 3 个不同的数据集。我只是在运行第一个数据集并将 epoch 数设置为 25，但我注意到我的验证损失在第 6 个 epoch 之后就在增加，这是否表明过度拟合，我做错了什么吗？

import pandas as pd
import tensorflow as tf
from keras.layers.core import Dense
from keras.layers.recurrent import GRU
from keras.models import Sequential
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from google.colab import files
from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback
tbc=TensorBoardColab() # Tensorboard


df10=pd.read_csv('/content/drive/My Drive/Isolation Forest/IF 10 PERCENT.csv',index_col=None)
df2_10= pd.read_csv('/content/drive/My Drive/2019 Dataframe/2019 10minutes IF 10 PERCENT.csv',index_col=None)

X10_train= df10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_train=X10_train.values

y10_train= df10['Power_kW']
y10_train=y10_train.values

X10_test= df2_10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_test=X10_test.values

y10_test= df2_10['Power_kW']
y10_test=y10_test.values




# scaling values for model


x_scale = MinMaxScaler()
y_scale = MinMaxScaler()

X10_train= x_scale.fit_transform(X10_train)
y10_train= y_scale.fit_transform(y10_train.reshape(-1,1))
X10_test=  x_scale.fit_transform(X10_test)
y10_test=  y_scale.fit_transform(y10_test.reshape(-1,1))


X10_train = X10_train.reshape((-1,1,12)) 
X10_test = X10_test.reshape((-1,1,12))







# creating model using Keras
model10 = Sequential()
model10.add(GRU(units=512, return_sequences=True, input_shape=(1,12)))
model10.add(GRU(units=256, return_sequences=True))
model10.add(GRU(units=256))
model10.add(Dense(units=1, activation='sigmoid'))
model10.compile(loss=['mse'], optimizer='adam',metrics=['mse']) 
model10.summary() 

history10=model10.fit(X10_train, y10_train, batch_size=256, epochs=25,validation_split=0.20, verbose=1, callbacks=[TensorBoardColabCallback(tbc)])


score = model10.evaluate(X10_test, y10_test)
print('Score: {}'.format(score))



y10_predicted = model10.predict(X10_test)
y10_predicted = y_scale.inverse_transform(y10_predicted)

y10_test = y_scale.inverse_transform(y10_test)

plt.plot( y10_predicted, label='Predicted')
plt.plot( y10_test, label='Measurements')
plt.legend()
plt.savefig('/content/drive/My Drive/Figures/Power Prediction 10 Percent.png')
plt.show()

Answer 1

LSTM（以及 GRU，尽管它们的结构更轻）因容易过拟合而臭名昭著。

减少每层（32（layer1）-64（layer2）中的单元数（输出大小）；您也可以完全消除最后一层。

其次，您使用的是激活“sigmoid”，但您的损失函数 + 度量是mse。

确保您的问题是regression 或classification 之一。如果确实是回归，那么最后一步的激活函数应该是'linear'。如果是分类，您应该将 loss_function 更改为 binary_crossentropy 并将您的指标更改为 'accuracy'。

因此，显示的情节暂时只是误导。如果您按照我的建议进行修改，但仍然得到这样的 train-val 损失图，那么我们可以确定您有过度拟合的情况。