Logstash - 删除包含 kv 值的日志答案

【问题标题】：Logstash - Drop logs containing kv valueLogstash - 删除包含 kv 值的日志
【发布时间】：2021-10-05 12:49:45
【问题描述】：

我没有成功尝试根据 kv 值字段的值删除日志。

 filter {
     if [type] == "cef" {
        mutate {
             add_field => { "tmp_message" => "%{message}" }
             split => ["message", "|"]
             add_field => { "version" => "%{message[0]}" }
             add_field => { "device_vendor" => "%{message[1]}" }
             add_field => { "device_product" => "%{message[2]}" }
             add_field => { "device_version" => "%{message[3]}" }
             add_field => { "sig_id" => "%{message[4]}" }
             add_field => { "sig_name" => "%{message[5]}" }
             add_field => { "sig_severity" => "%{message[6]}" }
        }
        kv {
             field_split => " "
             trim_value => "<>\[\],"
        }
        mutate {
            replace => { "message" => "%{tmp_message}" }
            remove_field => [ "tmp_message" ]
        }
      }
     if [FTNTFGTsrcintfrole_s] == "wan" {
       drop { }
}

[FTNTFGTsrcintfrole_s] 是被 kv 解析出来的键之一。如果键的值为“wan”，它应该删除日志。这不会发生。

如何过滤掉这些日志？

编辑：这是解析数据的示例

{
                    "dst" => "xxx.xxx.xxx.xxx",
              "FTNTFGTtz" => "+0000",
         "FTNTFGTsubtype" => "forward",
                "message" => "%{tmp_message}",
                   "host" => "xxx.xxx.xxx.xxx",
                    "spt" => "59975",
                   "type" => "cef",
 "deviceInboundInterface" => "ssl.root",
     "FTNTFGTdstintfrole" => "wan",
        "FTNTFGTduration" => "180",
      "FTNTFGTdstcountry" => "United",
        "FTNTFGTpolicyid" => "47",
      "FTNTFGTpolicytype" => "policy",
         "FTNTFGTpoluuid" => "801d40c2-3b60-51ea-d66a-293bf886d27e",
       "FTNTFGTeventtime" => "1633506791693710149",
"sourceTranslatedAddress" => "xxx.xxx.xxx.xxx",
                    "dpt" => "8253",
                    "app" => "udp/8253",
      "FTNTFGTpolicyname" => "xxxxxxxx",
                   "tags" => [
    [0] "fortigate",
    [1] "_mutate_error"
],
                    "act" => "accept",
           "FTNTFGTlogid" => "0000000013",
                     "in" => "64",
   "sourceTranslatedPort" => "59975",
         "FTNTFGTsentpkt" => "1",
        "FTNTFGTtrandisp" => "snat",
     "FTNTFGTsrcintfrole" => "wan",
               "@version" => "1",
         "FTNTFGTrcvdpkt" => "1",
       "deviceExternalId" => "xxxxx",
      "FTNTFGTauthserver" => "xxxxx",
             "@timestamp" => 2021-10-06T07:53:11.729Z,
      "FTNTFGTsrccountry" => "Reserved",
"deviceOutboundInterface" => "wan1",
                  "proto" => "17",
                    "out" => "48",
                    "src" => "xxx.xxx.xxx.xxx",
             "externalId" => "870512",
           "FTNTFGTlevel" => "notice",
              "FTNTFGTvd" => "root",
                  "duser" => "xxxxx",
                    "cat" => "traffic:forward",
          "FTNTFGTappcat" => "unscanned"
}

【问题讨论】：

您能否添加一些已解析数据的样本？
@YLR 是否有一个文件夹在发送之前存储解析的数据？我使用的 SIEM 实际上并没有显示原始数据包数据。
您可以删除过滤器并将stdout 输出为 json 格式，如果您不知道在哪里可以找到数据，这可能是最简单的方法。
@YLR 添加了一个示例

标签： logstash logstash-configuration

【解决方案1】：

感谢@YLR 和@Filip，我找到了答案。 SIEM 在创建字段时将“_s”添加到键名中，这让我相信那是原始键名，而这反过来又是我过滤的内容。在看到日志输出并意识到情况并非如此后，我更正了过滤器并且它起作用了。

【讨论】：