在 logstash 中使用 grok 解析多行 JSON 和数组

Question

您好，这是我第一次使用 Kibana。 我正在尝试将以下输入文件解析为 logstash 并将其放入弹性搜索以在 Kibana 中使用。

{
  "ASRtest": {
    "ASRHDR": "This is asr HDR",
    "ASRTestType": "DevTest",
    "Scenario": [
      {
        "ScenarioNumber": 1,
        "ScenarioName": "HTTP Validation",
        "ScenarioDescription": "Validate if the API alows access over HTTP",
        "ScExecutionStatus": "Execution Complete",
        "ScenarioStatus": "In-Complete",
        "ScenarioSeverity": false,
        "TestCase": [
          {
            "TestCaseNumber": 1,
            "TestCaseName": "HTTP Validation - using POST method ",
            "TcExecutionStatus": "Execution Error",
            "TcStatus": "NA",
            "TcSeverity": "NA"
          }
        ]
      },
      {
        "ScenarioNumber": 2,
        "ScenarioName": "Server Platform/Version Disclosure",
        "ScenarioDescription": "Validate if API disclose server information",
        "ScExecutionStatus": "Execution Complete",
        "ScenarioStatus": "Failure",
        "ScenarioSeverity": "Medium",
        "TestCase": [
          {
            "TestCaseNumber": 1,
            "TestCaseName": "Server Platform/Version Disclosure - using POST method ",
            "TcExecutionStatus": "Executed Successfully",
            "TcStatus": "Failure",
            "TcSeverity": "Medium"
          }
        ]
      }
    ]
  }
}

我希望输入中的所有字段都可用于仪表板图表。谁能解释如何将这个多行 JSON 文件解析为 logstash ==> elasticsearch。 我尝试了一个示例配置文件，但无法生成所需的输出。

Answer 1

您可能必须使用multilinecodec，它可以理想地允许将来自文件的多行消息加入到单个事件中。 input 可能看起来像这样：

input 
{   
    file 
    {
        codec => multiline
        {
            pattern => '^\{'
            negate => true
            what => previous                
        }
        path => ["path to your json file/.json"]
        start_position => "beginning"
        sincedb_path => "/dev/null"
        exclude => "*.gz"
    }
}

This 线程和SO 可能很有用。希望对您有所帮助！