高 val_acc，但在实践中精度低

【问题标题】：High val_acc, but low accuracy in practice高 val_acc，但在实践中精度低
【发布时间】：2018-03-10 20:46:22
【问题描述】：

我一直在按照this website 的代码训练一个可以区分 10 种猴子的 CNN。在训练阶段，我通过 3 个 epoch 获得了不错的结果：

Using TensorFlow backend.
Found 1097 images belonging to 10 classes.
Found 272 images belonging to 10 classes.
Epoch 1/3
46/46 [==============================] - 24s 521ms/step - loss: 1.4526 - acc: 0.5644 - val_loss: 0.5665 - val_acc: 0.9688
Epoch 2/3
46/46 [==============================] - 20s 445ms/step - loss: 0.5706 - acc: 0.8983 - val_loss: 0.4078 - val_acc: 0.9375
Epoch 3/3
46/46 [==============================] - 21s 449ms/step - loss: 0.3696 - acc: 0.9447 - val_loss: 0.4832 - val_acc: 0.8438

然后我编写了一个预测脚本，它查看一个目录并根据保存的模型进行预测。我尝试将它指向训练和验证文件夹，但两个结果在大约 16% 的时间里似乎都是正确的。为什么我的val_acc在训练时看起来很高，但当我实际使用模型进行预测时，它的表现却很差？

这是预测循环：

inputSize = (300, 400, 3)
for count, path in enumerate(allImages):
    img= cv2.imread(path)

    img = cv2.resize(img, (inputSize[0], inputSize[1]))
    img = np.reshape(img, (1, inputSize[0], inputSize[1], inputSize[2]))

    correct = path.split(os.path.sep)[-2] #get label from directory
    classes = labels[model.predict(img)[0].argmax(axis=0)]

    if args.show_predictions: #display prediction in a window
        fontSize = 0.8
        black = (0, 0, 0)
        white = (255, 255, 255)

        orig = cv2.resize(orig, (480, 480))
        cv2.putText(orig, 'prediction: {}'.format(classes), (0, 20),
            cv2.FONT_HERSHEY_SIMPLEX, fontSize, black, 10)
        cv2.putText(orig, 'prediction: {}'.format(classes), (0, 20),
            cv2.FONT_HERSHEY_SIMPLEX, fontSize, white, 2)

        cv2.putText(orig, 'answer: {}'.format(correct), (0, orig.shape[0] - 10),
            cv2.FONT_HERSHEY_SIMPLEX, fontSize, black, 10)
        cv2.putText(orig, 'answer: {}'.format(correct), (0, orig.shape[0] - 10),
            cv2.FONT_HERSHEY_SIMPLEX, fontSize, white, 2)
        cv2.imshow('prediction', orig)
        if cv2.waitKey(0) == ord('q'):
            break

    if args.verbose:
        print('prediction: {} --> answer: {}'.format(classes, correct))

    if classes == correct:
        totalCorrect += 1

    counter += 1
    if count%verbosity == 0:
        print('Predicting image {} out of {}'.format(count, len(allImages)))

print('total correct: {}/{} - {}%'.format(totalCorrect, counter, int(totalCorrect/counter * 100)))
cv2.destroyAllWindows()

基本上，它会从目录中读取标签并将其与我之前在脚本中创建的label 列表进行比较。

在使用 training 数据运行脚本后（我认为它会很高，因为它已经过训练了），我得到：

Predicting image 600 out of 1097
Predicting image 650 out of 1097
Predicting image 700 out of 1097
Predicting image 750 out of 1097
Predicting image 800 out of 1097
Predicting image 850 out of 1097
Predicting image 900 out of 1097
Predicting image 950 out of 1097
Predicting image 1000 out of 1097
Predicting image 1050 out of 1097
total correct: 161/1097 - 14%

是什么让训练中的准确率很高，但在实践中却很低？

编辑：以详细模式运行脚本：

Predicting image 0 out of 272
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: bald_uakari --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: bald_uakari --> answer: bald_uakari
prediction: japanese_macaque --> answer: bald_uakari
prediction: japanese_macaque --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: white_headed_capuchin --> answer: bald_uakari
prediction: bald_uakari --> answer: bald_uakari
prediction: white_headed_capuchin --> answer: bald_uakari
prediction: japanese_macaque --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: japanese_macaque --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: bald_uakari
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
Predicting image 50 out of 272
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: common_squirrel_monkey --> answer: black_headed_night_monkey
prediction: nilgiri_langur --> answer: common_squirrel_monkey
prediction: japanese_macaque --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: white_headed_capuchin --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: pygmy_marmoset --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: nilgiri_langur --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: white_headed_capuchin --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: nilgiri_langur --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: common_squirrel_monkey
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: pygmy_marmoset --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: pygmy_marmoset --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
Predicting image 100 out of 272
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: japanese_macaque --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: japanese_macaque
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: japanese_macaque --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: mantled_howler
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: white_headed_capuchin --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: white_headed_capuchin --> answer: nilgiri_langur
Predicting image 150 out of 272
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: nilgiri_langur
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: japanese_macaque --> answer: patas_monkey
prediction: nilgiri_langur --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: nilgiri_langur --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: nilgiri_langur --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: japanese_macaque --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: pygmy_marmoset --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: patas_monkey
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
Predicting image 200 out of 272
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: pygmy_marmoset --> answer: pygmy_marmoset
prediction: japanese_macaque --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: japanese_macaque --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: nilgiri_langur --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: common_squirrel_monkey --> answer: pygmy_marmoset
prediction: white_headed_capuchin --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: nilgiri_langur --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: japanese_macaque --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: nilgiri_langur --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: patas_monkey --> answer: silvery_marmoset
prediction: japanese_macaque --> answer: silvery_marmoset
prediction: japanese_macaque --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: japanese_macaque --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: silvery_marmoset
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
Predicting image 250 out of 272
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: japanese_macaque --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: pygmy_marmoset --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: white_headed_capuchin --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
prediction: common_squirrel_monkey --> answer: white_headed_capuchin
total correct: 41/272 - 15%

【问题讨论】：

  • 如果您可以在详细模式下运行代码并显示此行的输出会很有帮助：print('prediction: {} --> answer: {}'.format(classes, correct))
  • @KiraMichiru，刚刚将详细输出添加到问题中。看起来它大部分时间都是错误的。奇怪的是 val_acc 和 acc 都很高，但是当我尝试测量它时，这两个集合的准确度都很差：c
  • 输出显示您实际上得到了不同的预测，但是，预测远非随机。你几乎总是得到 common_squirrel_monkey 作为预测，这很奇怪。接下来，尝试直接输出model.predict(img)，看看是否也可以看到一个类被高度高估的相同模式。

标签： deep-learning keras conv-neural-network


【解决方案1】：

所以，事实证明，我使用了错误的通道输入模式。我使用的预训练网络称为 keras 库中的 resnet50，它使用 caffec 样式输入。

解决办法是从keras导入preprocess_image()，然后运行

img = preprocess_image(img)

另一个因素是我使用cv2.imread 加载图像。我不确定，但这可能会使用 resnet50 所需的错误通道格式。另一种方法是使用 keras 的 load_img() 函数。

结论：val_acc在训练时一般是正确的。如果您的预测准确度较差，则可能是数据输入问题！

【讨论】：

    猜你喜欢
    • 2017-10-05
    • 2012-10-20
    • 1970-01-01
    • 2016-02-18
    • 2021-06-16
    • 1970-01-01
    • 1970-01-01
    • 2023-03-30
    • 2017-12-22
    相关资源
    最近更新 更多
    热门标签
    Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode