loss: nan 何时建立共享单车模型

Question

我是新人，正在学习机器学习，如果提问的方式不太好，问题很简单，请多多包涵。

问题是我开发的模型以 nan 的形式返回损失，如果我做错了什么，请告诉我。以下是详细信息。

程序逻辑

import tensorflow as tf
import pandas as pd
# Reading the csv file from local drive as a dataframe
bike_df = pd.read_csv('C:\\Users\\HOME\\MLPythonPractice\\Data sets\\Bike-Sharing-Dataset\\day.csv')
bike_result_df = pd.read_csv('C:\\Users\\HOME\\MLPythonPractice\\Data sets\\Bike-Sharing-Dataset\\day.csv')

# Remove unwanted columns from the data frame
bike_df = bike_df.drop(columns=['instant','dteday','cnt'])
# shape of the dataframe
print(bike_df.shape)
# Exact attribute to see the columns of the dataframe
print(bike_df.columns)
# To know the type 
print(type(bike_df))
# To see the information of the dataframe
print(bike_df.info())
# Converting from dataframe to ndarray
bike_s = bike_df.values
print(type(bike_s))
print(bike_s.shape)
# Remove all the columns except cnt column which is result set
bike_result_df['cnt'] = bike_result_df['cnt'].values.astype(np.float64)  #converting to float
bike_result_df = bike_result_df['cnt']  # Removing all columns except cnt column
bike_result_s = bike_result_df.values   # Converting dataframe to ndarray
print(type(bike_result_s))
print(bike_result_s)
import numpy as np
print(type(bike_df))
print(bike_df.shape)
print(bike_result_df.shape)
#As the data frame is available, we will build the graph using keras (## are part of build graph)

## Initialise the sequential model
model = tf.keras.models.Sequential()
## Normalize the input data by creating a normalisation layer
model.add(tf.keras.layers.BatchNormalization(input_shape = (13,)))
## Add desnse layer for predition -- Keras declares weights and bias - dense(1) 1 here is expected value
model.add(tf.keras.layers.Dense(1))
# Compile the model - add loss and gradient descen optimiser
model.compile(optimizer='sgd',loss='mse')
print(type(bike_s))
print(type(bike_result_s))
print(bike_s.shape)
print(bike_result_s.shape)
print(bike_result_s)
# Execute the graph
model.fit(bike_s,bike_result_s,epochs=10)
model.save('models/bike_sharing_lr.h5')

---

我得到了输出

Epoch 1/10
731/731 [==============================] - 1s 895us/step - loss: nan     
Epoch 2/10
731/731 [==============================] - 0s 44us/step - loss: nan
Epoch 3/10
731/731 [==============================] - 0s 46us/step - loss: nan
Epoch 4/10
731/731 [==============================] - 0s 44us/step - loss: nan
Epoch 5/10
731/731 [==============================] - 0s 39us/step - loss: nan
Epoch 6/10
731/731 [==============================] - 0s 39us/step - loss: nan
Epoch 7/10
731/731 [==============================] - 0s 47us/step - loss: nan
Epoch 8/10
731/731 [==============================] - 0s 40us/step - loss: nan
Epoch 9/10
731/731 [==============================] - 0s 43us/step - loss: nan
Epoch 10/10
731/731 [==============================] - 0s 42us/step - loss: nan

Answer 1

为了防止你的渐变爆炸，你可以像这样剪辑它。

model.compile(optimizer=tf.keras.optimizers.SGD(clipnorm=1), loss='mse')

根据https://keras.io/optimizers/，设置clipnorm=1 允许梯度下降优化器控制梯度裁剪。所有参数梯度将被裁剪为最大范数 1。这可以防止您的损失函数发散。

另请参阅https://www.dlology.com/blog/how-to-deal-with-vanishingexploding-gradients-in-keras/，了解控制爆炸梯度的其他方法。

---

通过上述调整，损失函数不会发散，但也不会随着时间的推移而减少。我注意到您设置模型的方式很奇怪。批量标准化通常应遵循激活层。我不确定你为什么需要规范化你的输入，但你不应该为此使用BatchNormalize。如果您将模型更改为，

model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Dense(1, activation='relu'))

model.add(tf.keras.layers.BatchNormalization(input_shape = (13,)))

model.compile(optimizer='sgd', loss='mse')

你会得到一个更有意义的结果，损失函数值现在从大约 2000 万减少到 120000。