pytorch 中 tf.nn.softmax_cross_entropy_with_logits 的等价物是什么？

Question

我试图用 pytorch 复制一个用 tensorflow 编写的代码。我在 tensorflow 中遇到了一个损失函数，softmax_cross_entropy_with_logits。我在 pytorch 中寻找它的等价物，我发现了 torch.nn.MultiLabelSoftMarginLoss，虽然我不太确定它是正确的函数。我也不知道如何测量准确性当我使用这个损失函数并且网络末端没有 relu 层时，我的模型是我的代码：

# GRADED FUNCTION: compute_cost 

def compute_cost(Z3, Y):

    loss = torch.nn.MultiLabelSoftMarginLoss()    
    return loss(Z3,Y)


def model(net,X_train, y_train, X_test, y_test, learning_rate = 0.009,
          num_epochs = 100, minibatch_size = 64, print_cost = True):

    optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate)
    optimizer.zero_grad()

    total_train_acc=0

    for epoch in range(num_epochs):
        for i, data in enumerate(train_loader, 0):
            running_loss = 0.0

            inputs, labels = data

            inputs, labels = Variable(inputs), Variable(labels)

            Z3 = net(inputs)

            # Cost function
            cost = compute_cost(Z3, labels)

            # Backpropagation: Define the optimizer. 
            # Use an AdamOptimizer that minimizes the cost.

            cost.backward()
            optimizer.step()             

            running_loss += cost.item()

            # Measuring the accuracy of minibatch
            acc = (labels==Z3).sum()
            total_train_acc += acc.item()
            #Print every 10th batch of an epoch
            if epoch%1 == 0:
            print("Cost after epoch {} : 
            {:.3f}".format(epoch,running_loss/len(train_loader)))

Answer 1

使用torch.nn.CrossEntropyLoss()。它结合了softmax和交叉熵。来自文档：

此标准将 nn.LogSoftmax() 和 nn.NLLLoss() 组合在一个类中。

例子：

# define loss function
loss_fn = torch.nn.CrossEntropyLoss(reduction='mean')

# during training
for (x, y) in train_loader:
    model.train()
    y_pred = model(x) # your input `torch.FloatTensor`
    loss_val = loss_fn(y_pred, y)
    print(loss_val.item()) # prints numpy value

    optimizer.zero_grad()
    loss_val.backward()
    optimizer.step()

确保x 和y 的类型正确。通常转换是这样完成的：loss_fn(y_pred.type(torch.FloatTensor), y.type(torch.LongTensor)).

要测量准确性，您可以定义一个自定义函数：

def compute_accuracy(y_pred, y):
   if list(y_pred.size()) != list(y.size()):
      raise ValueError('Inputs have different shapes.',
                       list(y_pred.size()), 'and', list(y.size()))

  result = [1 if y1==y2 else 0 for y1, y2 in zip(y_pred, y)]

  return sum(result) / len(result)

并像这样使用两者：

model.train()
y_pred = model(x)

loss_val = loss_fn(y_pred.type(torch.FloatTensor), y.type(torch.LongTensor))
_, y_pred = torch.max(y_pred, 1)
accuracy_val = compute_accuracy(y_pred, y)
print(loss_val.item()) # print loss value
print(accuracy_val) # print accuracy value
# update step e.t.c

如果您的输入数据是 one-hot 编码的，您可以在使用 loss_fn 之前将其转换为常规编码：

_, targets = y.max(dim=1)
y_pred = model(x)
loss_val = loss_fn(y_pred, targets)