PyTorch 由于广播而给出不正确的结果

Question

我想用 PyTorch 运行一些神经网络实验，但是一个最小的测试用例给出了错误的答案。测试用例建立了一个简单的神经网络，其中包含两个输入变量和一个输出变量，该变量只是输入的总和，并尝试将其作为回归问题进行学习；我希望它收敛于零均方误差，但它实际上收敛于 0.165。这可能是因为警告消息中提到的问题；我该如何解决？

代码：

import torch
import torch.nn as nn

# data
Xs = []
ys = []
n = 10
for i in range(n):
    i1 = i / n
    for j in range(n):
        j1 = j / n
        Xs.append([i1, j1])
        ys.append(i1 + j1)

# torch tensors
X_tensor = torch.tensor(Xs)
y_tensor = torch.tensor(ys)

# hyperparameters
in_features = len(Xs[0])
hidden_size = 100
out_features = 1
epochs = 500

# model
class Net(nn.Module):
    def __init__(self, hidden_size):
        super(Net, self).__init__()
        self.L0 = nn.Linear(in_features, hidden_size)
        self.N0 = nn.ReLU()
        self.L1 = nn.Linear(hidden_size, 1)

    def forward(self, x):
        x = self.L0(x)
        x = self.N0(x)
        x = self.L1(x)
        return x


model = Net(hidden_size)
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.1)

# train
print("training")
for epoch in range(1, epochs + 1):
    # forward
    output = model(X_tensor)
    cost = criterion(output, y_tensor)

    # backward
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()

    # print progress
    if epoch % (epochs // 10) == 0:
        print(f"{epoch:6d} {cost.item():10f}")
print()

output = model(X_tensor)
cost = criterion(output, y_tensor)
print("mean squared error:", cost.item())

输出：

training
C:\Users\russe\Anaconda3\envs\torch2\lib\site-packages\torch\nn\modules\loss.py:445: UserWarning: Using a target size (torch.Size([100])) that is different to the input size (torch.Size([100, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
    50   0.167574
   100   0.165108
   150   0.165070
   200   0.165052
   250   0.165039
   300   0.165028
   350   0.165020
   400   0.165013
   450   0.165009
   500   0.165006

mean squared error: 0.1650056540966034

还有消息：

UserWarning：使用与输入大小 (torch.Size([100, 1])) 不同的目标大小 (torch.Size([100]))。由于广播，这可能会导致不正确的结果。请确保它们具有相同的尺寸。

Answer 1

您将更具体地说明哪些张量（X 或 Y），但我们可以使用 torch.view function 重塑我们的张量。

例如：

Y_tensor = torch.tensor(Ys)
print(Y_tensor.shape)
>> torch.Size([5])

new_shape = (len(Ys), 1)
Y_tensor = Y_tensor.view(new_shape)
print(Y_tensor.shape)
>> torch.Size([5, 1])

但是，我怀疑这种广播行为是否会导致您遇到准确性问题。