Logstash grok 过滤器 Apache 模式

Question

这是一个示例 Apache Tomcat 日志：

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2

我想捕获以下变量

09/Sep/2019:00:08:02 +0200

/en/search-results?p_p_id=portal2....

35S64857F6860FFC0F60B5B47A97E18

你能帮我解决这个问题吗？我只想索引那些并删除其他的，这可能吗？谢谢

Answer 1

使用这个 grok 模式：

%{GREEDYDATA:field1} %{IP:ip1} - - \[%{GREEDYDATA:date}] \"%{WORD:method} %{GREEDYDATA:request}" %{WORD:numbers} %{WORD:numbers} %{WORD:numbers} - - %{WORD:last_parameter}

输入：

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2

输出：

{
  "field1": [
    [
      "portal.portal.some.thing.int:8443"
    ]
  ],
  "ip1": [
    [
      "13.233.220.113"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "13.233.220.113"
    ]
  ],
  "date": [
    [
      "09/Sep/2019:00:08:02 +0200"
    ]
  ],
  "method": [
    [
      "GET"
    ]
  ],
  "request": [
    [
      "/en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1"
    ]
  ],
  "numbers": [
    [
      "230",
      "334734",
      "6261"
    ]
  ],
  "last_parameter": [
    [
      "35S64857F6860FDFC0F60B5B47A97E18"
    ]
  ]
}

你想要的字段是：

• 日期
• 请求
• last_parameter

您可以在 mutate 过滤器中使用 remove field 删除其他字段。