PyTorch DataLoader -“IndexError：0 维张量的索引过多”答案

【问题标题】：PyTorch DataLoader - "IndexError: too many indices for tensor of dimension 0"PyTorch DataLoader -“IndexError：0 维张量的索引过多”
【发布时间】：2019-11-06 18:53:06
【问题描述】：

我正在尝试实现一个 CNN 来识别 MNIST 数据集中的数字，而我的代码在数据加载过程中出现了错误。我不明白为什么会这样。

import torch
import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5), (0.5))
])

trainset = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=20, shuffle=True, num_workers=2)

testset = torchvision.datasets.MNIST(root='./data', train=False, download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=20, shuffle=False, num_workers=2)

for i, data in enumerate(trainloader, 0):
    inputs, labels = data[0], data[1]

错误：

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-6-b37c638b6114> in <module>
      2 
----> 3     for i, data in enumerate(trainloader, 0):
      4         inputs, labels = data[0], data[1]

# ...

IndexError: Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 99, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/opt/conda/lib/python3.6/site-packages/torchvision/datasets/mnist.py", line 95, in __getitem__
    img = self.transform(img)
  File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 61, in __call__
    img = t(img)
  File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 164, in __call__
    return F.normalize(tensor, self.mean, self.std, self.inplace)
  File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 208, in normalize
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
IndexError: too many indices for tensor of dimension 0

【问题讨论】：

标签： python machine-learning pytorch

【解决方案1】：

问题在于mean 和std 必须是序列（例如，元组），因此您应该在值之后添加一个逗号：

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

注意(0.5) 和(0.5,) 之间的区别。您可以查看here 这些值的使用方式。如果您应用相同的过程，您会看到：

import torch

x1 = torch.as_tensor((0.5))
x2 = torch.as_tensor((0.5,))

print(x1.shape, x1.ndim)  # output: torch.Size([]) 0
print(x2.shape, x2.ndim)  # output: torch.Size([1]) 1

也许你不知道，但它们在 Python 中也是不同的：

type((0.5))   # <type 'float'>
type((0.5,))  # <type 'tuple'>

【讨论】：

【解决方案2】：

检查 trainset 是否不为空，简单的打印输出，trainloader 也一样，如果仍然无法正常工作，我更喜欢手动加载 mnist

def load_mnist_labels(fnlabel):
f = gzip.open(fnlabel, 'rb')
f.read(8)
return np.frombuffer(f.read(), dtype = np.uint8)

def load_mnist_images(fnlabel):
f = gzip.open(fnlabel, 'rb')
f.read(16)
return np.frombuffer(f.read(), dtype = np.uint8)

【讨论】：