wdio cucumber json 记者在 Jira Xray 中的错误测试结果答案

【问题标题】：Wrong tests results in Jira Xray from wdio cucumber json reporterwdio cucumber json 记者在 Jira Xray 中的错误测试结果
【发布时间】：2021-02-08 21:53:34
【问题描述】：

我们有一个与 Cucumber 测试相关联的故事，我们在测试执行中执行此测试。 Test Execution 的功能文件如下所示：

@ABC-164
@ABC-163
Feature: Homepage test
    @ABC-162
    Scenario: Homepage test
        Feature: Go to homepage
            Scenario: Homepage
                Given I open the url "https://example.com/"
                Then I expect that the url is "https://google.com"

第一个总是通过，第二个失败。我们正在从 wdio cucumber json 报告器生成 JSON 报告，您可以在下面看到：

[
  {
    "keyword": "Feature",
    "type": "feature",
    "description": "",
    "line": 3,
    "name": "Homepage test",
    "uri": "Can not be determined",
    "tags": [
      {
        "type": "Tag",
        "location": { "line": 1, "column": 1 },
        "name": "@ABC-164"
      },
      {
        "type": "Tag",
        "location": { "line": 2, "column": 1 },
        "name": "@ABC-163"
      }
    ],
    "elements": [
      {
        "keyword": "Scenario",
        "type": "scenario",
        "description": "\t\tFeature: Go to homepage",
        "name": "Homepage test",
        "tags": [
          { "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
          { "name": "@ABC-163", "location": { "line": 2, "column": 1 } },
          { "name": "@ABC-162", "location": { "line": 6, "column": 2 } }
        ],
        "id": "homepage-test;homepage-test",
        "steps": [
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 2000000 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          }
        ]
      },
      {
        "keyword": "Scenario",
        "type": "scenario",
        "description": "",
        "name": "Homepage",
        "tags": [
          { "name": "@ABC-164", "location": { "line": 1, "column": 1 } },
          { "name": "@ABC-163", "location": { "line": 2, "column": 1 } }
        ],
        "id": "homepage-test;homepage",
        "steps": [
          {
            "arguments": [],
            "keyword": "Before",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Given",
            "name": "I open the url \"https://example.com/\"",
            "result": { "status": "passed", "duration": 587000000 },
            "line": 12,
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "Then",
            "name": "I expect that the url is \"https://google.com\"",
            "result": {
              "status": "failed",
              "duration": 11000000,
              "error_message": "Error: expect(received).toEqual(expected)"
            },
            "line": 13,
            "match": { "location": "can not be determined with webdriver.io" }
          },
          {
            "arguments": [],
            "keyword": "After",
            "name": "Hook",
            "result": { "status": "passed", "duration": 0 },
            "line": "",
            "match": { "location": "can not be determined with webdriver.io" }
          }
        ]
      }
    ],
    "id": "homepage-test",
    "metadata": {
      "browser": { "name": "chrome", "version": "88.0.4324.150" },
      "device": "Device name not known",
      "platform": { "name": "osx", "version": "Version not known" }
    }
  }
]

Xray 仅从第一个对象的 Before Hooks 中读取状态，而完全忽略其余状态。

当我们使用 Jenkins 插件以及通过 curl 直接发送时会出现问题：curl -k -H "Content-Type: application/json" -X POST -u login:pass --data @result.json https://jira/rest/...

任何帮助将不胜感激。

【问题讨论】：

标签： json cucumber webdriver-io jira-xray

【解决方案1】：

场景下不能有功能。这是一个语法错误，虽然 Cucumber 可能无法正确报告它。

【讨论】：

【解决方案2】：

正如 Aslak 在评论中提到的，.feature 文件语法不正确。正确的 .feature 文件类似于前面的示例。另请注意，您拥有的标签 (@ABC-xxx) 可能需要一些前缀，例如 REQ 或 TEST，具体取决于您的 settings。

顺便说一句，目前尚不清楚您是否要从 Xray 导出测试执行以生成 .feature 文件，但我认为情况并非如此。只是强调一下，为了根据现有测试（Cucumber Scenario/Scenario Outlines）报告结果，它们必须事先已经存在于 Xray 中。

@ABC-164
@ABC-163
Feature: Homepage test
    @ABC-162
    Scenario: Homepage
      Given I open the url "https://example.com/"
      Then I expect that the url is "https://google.com"*

【讨论】：

【解决方案3】：

对于那些会遇到同样问题的人。我们设法找到了解决方案。非常感谢您的回答！

看起来我们在 Jira 中有一个旧的功能文件结构（当时我们没有故事和先决条件）。当我们开始正确使用它时：将故事链接到测试并添加前提条件，然后 Xray 创建了双倍的场景部分。所以我们从 Cucumber Tests 中删除了多余的部分，创建了适当的 Pre-condition，现在特征文件就这样正确地创建了：

@ABC-169
@ABC-131
Feature: Main feature title

    Background:
        #@ABC-166
        Given I open the url "https://example.com/"
        Then I expect that the url is "https://google.com"

    @ABC-132 @ABC-168 @ABC-141
    Scenario: Main scenario
        Given Lorem ipsum
        Then Et sit amet

【讨论】：