mxnet 训练损失永远不会改变，但准确性会波动答案

【问题标题】：mxnet training loss never changes but accuracy oscillatesmxnet 训练损失永远不会改变，但准确性会波动
【发布时间】：2018-09-04 09:38:58
【问题描述】：

我正在使用mxnet训练一个VQA模型，输入为(6244,)向量，输出为单个标签

在我的 epoch 中，loss 从未改变，但准确率在小范围内波动，前 5 个 epoch 是

Epoch 1. Loss: 2.7262569132562255, Train_acc 0.06867348986554285
Epoch 2. Loss: 2.7262569132562255, Train_acc 0.06955649207304837
Epoch 3. Loss: 2.7262569132562255, Train_acc 0.06853301224162152
Epoch 4. Loss: 2.7262569132562255, Train_acc 0.06799116997792494
Epoch 5. Loss: 2.7262569132562255, Train_acc 0.06887417218543046

这是一个多类分类问题，每个答案标签代表一个类，所以我使用softmax作为最终层和交叉熵来评估损失，它们的代码如下

那么为什么损失永远不会改变？...我只是直接从cross_entropy得到if

trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.01})
loss = gluon.loss.SoftmaxCrossEntropyLoss()

epochs = 10
moving_loss = 0.
best_eva = 0
for e in range(epochs):
    for i, batch in enumerate(data_train):
        data1 = batch.data[0].as_in_context(ctx)
        data2 = batch.data[1].as_in_context(ctx)
        data = [data1, data2]
        label = batch.label[0].as_in_context(ctx)
        with autograd.record():
            output = net(data)
            cross_entropy = loss(output, label)
            cross_entropy.backward()
        trainer.step(data[0].shape[0])

        moving_loss = np.mean(cross_entropy.asnumpy()[0])

    train_accuracy = evaluate_accuracy(data_train, net)
    print("Epoch %s. Loss: %s, Train_acc %s" % (e, moving_loss, train_accuracy))

eval函数如下

def evaluate_accuracy(data_iterator, net, ctx=mx.cpu()):
numerator = 0.
denominator = 0.
metric = mx.metric.Accuracy()
data_iterator.reset()
for i, batch in enumerate(data_iterator):
    with autograd.record():
        data1 = batch.data[0].as_in_context(ctx)
        data2 = batch.data[1].as_in_context(ctx)
        data = [data1, data2]
        label = batch.label[0].as_in_context(ctx)
        output = net(data)

    metric.update([label], [output])
return metric.get()[1]

【问题讨论】：

标签： python machine-learning computer-vision mxnet

【解决方案1】：

在 mxnet 论坛here 上提出和回答的问题。计算精度时无需使用autograd.record范围记录计算图。试试吧：

def evaluate_accuracy(data_iterator, net, ctx=mx.cpu()):
    metric = mx.metric.Accuracy()
    data_iterator.reset()
    for i, batch in enumerate(data_iterator):
        data1 = batch.data[0].as_in_context(ctx)
        data2 = batch.data[1].as_in_context(ctx)
        data = [data1, data2]
        label = batch.label[0].as_in_context(ctx)
        output = net(data)
        metric.update([label], [output])
    return metric.get()[1]

【讨论】：

错误是每个时期开始时训练循环中缺少“data.iterator.reset()”