【问题标题】：Get LUIS training evaluation metrics & result using API使用 API 获取 LUIS 训练评估指标和结果
【发布时间】：2021-12-02 16:10:02
【问题描述】：

我正在开发一个使用 API 调用与 LUIS 集成的解决方案。

我找不到从仪表板获取信息的 API，例如不正确的预测和不明确的预测。到目前为止，我只找到了这个 API：

{ENDPOINT}/luis/webapi/v2.0/apps/{APP_ID}/versions/{VERSION}/statsmetadata

但这还不够，因为我会得到更详细的信息，例如仪表板中显示的信息：

【问题讨论】：

请提供足够的代码，以便其他人更好地理解或重现问题。

标签： azure api azure-language-understanding

【解决方案1】：

statsmetadata 端点是显示饼图数字的正确操作。

样本

这是我的一个 LUIS 应用中的示例

仪表板显示

仪表板提供以下详细信息：

数据来源

StatsMetadata 端点提供所有详细信息。这是拨打的电话：https://westeurope.api.cognitive.microsoft.com//luis/webApi/v2.0/apps/*MyAppId*/versions/*MyAppVersion*/statsmetadata

JSON 结果：

{
    "appVersionUtterancesCount": 8505,
    "modelsMetadata": [{
        "modelId": "IdOfModel1,
        "modelName": "modelName1",
        "utterancesCount": 85,
        "misclassifiedUtterancesCount": 7,
        "ambiguousUtterancesCount": 19,
        "misclassifiedAmbiguousUtterancesCount": 7
    }, {
        "modelId": "IdOfModel2",
        "modelName": "modelName2",
        "utterancesCount": 402,
        "misclassifiedUtterancesCount": 11,
        "ambiguousUtterancesCount": 32,
        "misclassifiedAmbiguousUtterancesCount": 11
    }, {
        "modelId": "IdOfModel3",
        "modelName": "modelName3",
        "utterancesCount": 293,
        "misclassifiedUtterancesCount": 9,
        "ambiguousUtterancesCount": 42,
        "misclassifiedAmbiguousUtterancesCount": 9
    }, {
        "modelId": "IdOfModel4",
        "modelName": "modelName4",
        "utterancesCount": 58,
        "misclassifiedUtterancesCount": 3,
        "ambiguousUtterancesCount": 5,
        "misclassifiedAmbiguousUtterancesCount": 3
    }, {
        "modelId": "IdOfModel5",
        "modelName": "modelName5",
        "utterancesCount": 943,
        "misclassifiedUtterancesCount": 7,
        "ambiguousUtterancesCount": 103,
        "misclassifiedAmbiguousUtterancesCount": 7
    }, {
        "modelId": "IdOfModel6",
        "modelName": "modelName6",
        "utterancesCount": 266,
        "misclassifiedUtterancesCount": 9,
        "ambiguousUtterancesCount": 70,
        "misclassifiedAmbiguousUtterancesCount": 9
    }, {
        "modelId": "IdOfModel7",
        "modelName": "modelName7",
        "utterancesCount": 2441,
        "misclassifiedUtterancesCount": 68,
        "ambiguousUtterancesCount": 180,
        "misclassifiedAmbiguousUtterancesCount": 67
    }, {
        "modelId": "IdOfModel8",
        "modelName": "modelName8",
        "utterancesCount": 704,
        "misclassifiedUtterancesCount": 40,
        "ambiguousUtterancesCount": 154,
        "misclassifiedAmbiguousUtterancesCount": 40
    }, {
        "modelId": "IdOfModel9",
        "modelName": "modelName9",
        "utterancesCount": 288,
        "misclassifiedUtterancesCount": 9,
        "ambiguousUtterancesCount": 65,
        "misclassifiedAmbiguousUtterancesCount": 9
    }, {
        "modelId": "IdOfModel10",
        "modelName": "modelName10",
        "utterancesCount": 18,
        "misclassifiedUtterancesCount": 1,
        "ambiguousUtterancesCount": 1,
        "misclassifiedAmbiguousUtterancesCount": 1
    }, {
        "modelId": "IdOfModel11",
        "modelName": "modelName11",
        "utterancesCount": 444,
        "misclassifiedUtterancesCount": 12,
        "ambiguousUtterancesCount": 50,
        "misclassifiedAmbiguousUtterancesCount": 12
    }, {
        "modelId": "IdOfModel12",
        "modelName": "modelName12",
        "utterancesCount": 10,
        "misclassifiedUtterancesCount": 8,
        "ambiguousUtterancesCount": 8,
        "misclassifiedAmbiguousUtterancesCount": 8
    }, {
        "modelId": "IdOfModel13",
        "modelName": "modelName13",
        "utterancesCount": 58,
        "misclassifiedUtterancesCount": 9,
        "ambiguousUtterancesCount": 21,
        "misclassifiedAmbiguousUtterancesCount": 9
    }, {
        "modelId": "IdOfModel14",
        "modelName": "None",
        "utterancesCount": 2495,
        "misclassifiedUtterancesCount": 194,
        "ambiguousUtterancesCount": 428,
        "misclassifiedAmbiguousUtterancesCount": 178
    }],
    "intentsCount": 14,
    "entitiesCount": 7,
    "phraseListsCount": 7,
    "patternsCount": 0,
    "trainingTime": "00:00:53.2732860",
    "lastTrainDate": "2019-10-22T13:25:03Z",
    "misclassifiedUtterancesCount": 387,
    "ambiguousUtterancesCount": 1178,
    "misclassifiedAmbiguousUtterancesCount": 370
}

API 数据计算 - 饼图

不正确的预测

这里是错误分类的话语数除以话语总数

公式：misclassifiedUtterancesCount / appVersionUtterancesCount
检查：387 / 8505 = 0.04550 = 4.6%

不清楚的预测

这里是歧义（但不是错误分类的歧义，因为它们已经计入不正确的预测）话语的数量除以话语的总数

公式：(ambiguousUtterancesCount - misclassifiedAmbiguousUtterancesCount) / appVersionUtterancesCount
检查：(1178 - 370) / 8505 = 808 / 8505 = 0.09500 = 9.5%

正确的预测

这是……剩下的！所以：公式：(appVersionUtterancesCount - misclassifiedUtterancesCount - (ambiguousUtterancesCount - misclassifiedAmbiguousUtterancesCount)) / appVersionUtterancesCount 检查：(8505 - 387 - (1178 - 370)) / 8505 = (8505 - 387 - 808) / 8505 = 7310 / 8505 = 0.8594 = 85.9%

API 数据计算 - 右侧面板详细信息

数据不平衡

是utterancesCount数量最多的模型，但是不知道有没有阈值比例用来只保留前N个模型

不正确的预测

按型号，检查以下比率：misclassifiedUtterancesCount / utterancesCount，并保留比率最高的。

不清楚的预测

按型号，检查以下比率：(ambiguousUtterancesCount - misclassifiedAmbiguousUtterancesCount) / utterancesCount，并保留比率最高的。

详情

在显示仪表板之前，您可以直接在导航器开发人员工具中检查 LUIS 门户正在调用此端点。

【讨论】：