Apache Log grok 模式答案

【问题标题】：Apache Log grok patternApache Log grok 模式
【发布时间】：2021-02-10 10:58:48
【问题描述】：

任何人都可以为下面的日志示例提供 grok 模式吗？

85.85.85.85 webmail.company.com "CN=First Last/O=Company/C=CZ" [14/Dec/2020:05:58:18 +0100] "GET /mail/User. nsf/iNotes/Proxy/?OpenDocument&Form=s_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName;($Inbox),UnreadCountInfo;1,SearchSort;DateD,s_UsingHttps;1,noPI;1&TZType=UTC&Start=1&Count=23&resortdescending=6 HTTP/1.1" 200 2054 "https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9_x&charset=UTF-8&charset=-UTF-8&charset=-8 safari&pt" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" 125 INOTES_LOGIN_ID=First%20Last; Shimmer=SI_TLM:20210209T072811%2C40Z&ST_Counter:3&LAO:mail&SAB:1&CS_TLM:20210209T072831%2C15Z&V_TLM:20210210T080147%2C82Z&DMS:5&ui:X&MOTLM:20210129T113159%2C00Z&DBQS:1503571%2C%207168000%2C%206963200%2C%200%2C%201503571&SPRKL:1&KOSCZ: GTB&FISD：1； INOTES_LOGIN_ID=第一个%20最后一个； DWAShared=0; DWAMode=0; INOTES_LOGIN_ID=第一个%20最后一个； DWAShared=0; DWAMode=0; LtpaToken2=FpoGJJz33bYLI+CtWy6OlIgoTJouNGEiduvxvQbcN8HRI7K6LThCsb1Dl8CzN72Zi05RGOUmQRMiOQcTk1norKHi6SbkEGI6GlXzjSIweBRSc8c+XPyAwA44PKPbu3WzrPfR0+uoC0sgTPvochvQ/VfPL/sSaqUFoRswRwyI+UeaOwTs/DvKiWLCpiKrVkFk3SmDjrxPBHb/WiL5nDkpp8Dsjjxnlo4vpx7BdOoVNai1jybvHkW28KXxkb21o8SSpmU7ZFdHyZFjDWCYuuCVOx7asV/q4a3lWdxlPfWdPcUguHML+xDmsrMPm6fTUSKeKIKdQEPr6VDmitBi7Z5URIlkRrUyslkTcc28y6fQir3Y20Hc9TmOvwaBlG/ehnpv; LtpaToken=0x4JJ4oWKojdqoz08Ng+MRUkkJq2vYGLGN9lp8HL8FxbD+xnivE7qzCzf92Q6x5OAPOBFRNgxd3Qg225zLwnJFWO0lGeIweH8VDgyWOMImNe6E9z9HBnQAN43vQ2uwtpv3X5E5DN0oLIPKLxAkqsHUDJqJ0SE6NZ6UnfLoR82JyjZVC/s6QEov5DNdpAY/o2Gxh0vWmE+wuQGuCh4mVCIP9KU/dbX4F0Ld9JEExzIpkdzKELibU2Akov0Krv0eWADSV++m/5ECLpaf6N6/VzkZEkt5XoOoL6OD/6ni4zojvo3O+X9Bn7Mdk2MnsQ1AccIohj5eN8Oi81QbD0a9b7jw==; ShimmerS=ET:20210210T114045%2c00Z&R:0&AT:M" "D:/Lotus/Domino/Data/mail/User.nsf"

我需要的是 Client IP (85.85.85.85), VirtualHostname (webmail / webmail.company.com) , User ( CN= 之后的部分，First Last)，Time (14/Dec/2020:05:58:18)，URL (GET /mail/User.nsf/iNotes/ Proxy/?OpenDocument&Form=s_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName; ... ) 和 设备信息 ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) ) Chrome/87.0.4280.88 Safari/537.36" )

我知道它应该从下面开始，但是无论如何我无法让用户名继续 [%{HTTPDATE:timestamp}] 并且下一个可能是 "(?:%{WORD:verb} %{NOTSPACE :request} 并且不确定如何获取设备信息。任何帮助将不胜感激！

%{IPORHOST:clientip} %{WORD:VirtualHost} ???

【问题讨论】：

标签： apache elasticsearch logging filebeat grok

【解决方案1】：

由于您已经自定义了日志格式，因此您必须构建自己的 grok 以匹配日志。你可以使用https://grokdebug.herokuapp.com/来调试你要使用的模式，你可以从https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns复制一些模式

【讨论】：

【解决方案2】：

已解决：

%{IPORHOST:clientip} %{IPORHOST:destination.domain} "CN=%{DATA:username}" [%{HTTPDATE:apache.access.time}] "(?:%{WORD:http.request .method} %{DATA:url.original} HTTP/%{NUMBER:http.version}|-)？” %{NUMBER:http.response.status_code:long} (?:%{NUMBER:http.response.body.bytes:long}|-) ("%{DATA:http.request.referrer}") ("%{数据：user_agent.original}")

【讨论】：