模型返回一个 Nan 值

Question

我试图构建一个具有 4 个输入节点/特征和一个输出特征 (0/1) 的神经网络。我编写了这段代码，它运行但在训练模型时返回 NaN。我也进行了调试，权重和偏差在它们通过模型之前都很好。

从我目前的搜索结果来看，这可能是我传递数据的方式存在的问题。 我的输入数据是：tensor([[0.0000e+00, 0.0000e+00, 0.0000e+00, 1.5340e+00], [1.5000e+01, 1.0000e-01, 2.4210e+00, 3.0000e+01], [3.0000e+00, 2.2000e-01, 2.2000e-01, 4.5000e+01], ..., [1.0000e+00, 2.0000e-02, 2.0000e-02, 1.5000e+01], [6.0000e+00, 2.0000e-01, 2.0000e-01, 1.5000e+01], [1.7000e+01, 5.2400e-01, 5.2400e-01, 2.0000e+00]], dtype=torch.float64)

import torch
from torchvision import datasets, transforms
import pandas as pd
import numpy as np
from torch.autograd import Variable
# Import tensor dataset & data loader
from torch.utils.data import TensorDataset, DataLoader
from torch import nn, optim
import torch.nn.functional as F

file = pd.read_csv('ks-projects-201801.csv')
array = np.array(file.values)
result = np.empty(len(array))
input_data = np.empty((len(array), 4))
for i in range(len(array)):
    input_data[i] = np.array([array[i][10], array[i][12]/1000, array[i][13]/1000, array[i][14]/1000])
    if array[i][9] == 'successful':
        result[i] = 1
    else:
        result[i] = 0


input_node = Variable(torch.from_numpy(input_data))
output = torch.from_numpy(result)
print(input_node)
print(output)

train_ds = TensorDataset(input_node.squeeze(), output.squeeze())

batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)

这是实际的模型和训练

model = nn.Linear(4, 1)
print(model.weight)
print(model.bias)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.003)

epochs = 5
model = model.double()
for e in range(epochs):
    running_loss = 0
    for xb, yb in train_dl:
        optimizer.zero_grad()
        res = model(xb)
        loss = criterion(res, yb)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
    else:
        print(f"model : {loss}")

这会打印出 model: nan 用于每个 epoch 并终止。我对 pytorch 很陌生，不知道如何处理这个问题。

Answer 1

如果您看到 NaN 丢失，请尝试梯度裁剪和数据归一化。标准化数据是必须的（即标准化输入数据，使得均值 = 0 和方差 =1）