用于 AWS Cloudtrail json 日志的 Splunk 道具配置答案

【问题标题】：Splunk props config for AWS Cloudtrail json logs用于 AWS Cloudtrail json 日志的 Splunk 道具配置
【发布时间】：2023-08-28 00:19:01
【问题描述】：

我需要提取从 S3 提取的 AWS cloudtrail 日志。这些文件包含单个 json 有效负载，其中包含各个 cloudtrail 事件。但是 splunk 无法识别单个事件，也无法正确拆分。它只是一个单一事件的一大块

每个文件都包含这种格式：

  "Records": [
    {
      "apiVersion": "2012-06-01",
      "awsRegion": "us-west-1",
      "eventID": "c-c245-2c4-32v6-vfff",
      "eventName": "DescribeLoadBalancers",
      "eventSource": "elasticloadbalancing.amazonaws.com",
      "eventTime": "2019-11-30T18:15:33Z",
      "eventType": "AwsApiCall",
      "eventVersion": "1.05",
      "recipientAccountId": "redacted",
      "requestID": "2xc454xc-2345-234cv5-2345",
      "requestParameters": null,
      "responseElements": null,
      "sourceIPAddress": "1.1.1.1",
      "userAgent": "aws-sdk-ruby3/3.75.0 jruby/2.3.3 java aws-sdk-elasticloadbalancing/1.19.0 cloudhealth",
      "userIdentity": {
        "accessKeyId": "redacted",
        "accountId": "redacted",
        "arn": "arn:aws:sts::redacted:assumed-role/team/AssumeRoleSession",
        "principalId": "redacted:AssumeRoleSession",
        "sessionContext": {
          "attributes": {
            "creationDate": "2019-11-30T17:45:06Z",
            "mfaAuthenticated": "false"
          },
          "sessionIssuer": {
            "accountId": "redacted",
            "arn": "arn:aws:iam::redacted:team/company",
            "principalId": "redacted",
            "type": "Role",
            "userName": "redacted"
          },
          "webIdFederationData": {}
        },
        "type": "AssumedRole"
      }
    },{
      "apiVersion": "2012-06-01",
      "awsRegion": "us-west-1",
      "eventID": "c-c245-2c4-32v6-vfff",
      "eventName": "DescribeLoadBalancers",
      "eventSource": "elasticloadbalancing.amazonaws.com",
      "eventTime": "2019-11-30T18:16:33Z",
      "eventType": "AwsApiCall",
      "eventVersion": "1.05",
      "recipientAccountId": "redacted",
      "requestID": "2xc454xc-2345-234cv5-2345",
      "requestParameters": null,
      "responseElements": null,
      "sourceIPAddress": "1.1.1.1",
      "userAgent": "aws-sdk-ruby3/3.75.0 jruby/2.3.3 java aws-sdk-elasticloadbalancing/1.19.0 cloudhealth",
      "userIdentity": {
        "accessKeyId": "redacted",
        "accountId": "redacted",
        "arn": "arn:aws:sts::redacted:assumed-role/team/AssumeRoleSession",
        "principalId": "redacted:AssumeRoleSession",
        "sessionContext": {
          "attributes": {
            "creationDate": "2019-11-30T17:45:06Z",
            "mfaAuthenticated": "false"
          },
          "sessionIssuer": {
            "accountId": "redacted",
            "arn": "arn:aws:iam::redacted:role/team",
            "principalId": "redacted",
            "type": "Role",
            "userName": "redacted"
          },
          "webIdFederationData": {}
        },
        "type": "AssumedRole"
      }
    }
  ]
}

我的道具是这样的

[cloudtrail]
KV_MODE = json

【问题讨论】：

标签： amazon-web-services splunk amazon-cloudtrail

【解决方案1】：

一些进一步的谷歌搜索和反复试验导致这个道具配置似乎正确地破坏了事件

[cloudtrail]
KV_MODE = json
SHOULD_LINEMERGE=false
LINE_BREAKER=((?<=}),(?={)|[\r\n]+)
SEDCMD-remove_prefix=s/{"Records":\[//g
SEDCMD-remove_suffix=s/\]}//g

【讨论】：