允许 AWS Athena 查询 AWS 访问日志答案

【问题标题】：Allow AWS Athena to query AWS access logs允许 AWS Athena 查询 AWS 访问日志
【发布时间】：2021-03-11 20:28:25
【问题描述】：

我将我的 nginx 访问日志存储在 AWS S3 中，我想使用 Athena 来查询这些日志。

我可以成功创建数据库表，但是当我尝试在 Athena 控制台中查询日志时，出现以下错误：

Your query has the following error(s):
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XXXXX; S3 Extended Request ID: XXXXXX=; Proxy: null), S3 Extended Request ID: XXXXXX= (Path: s3://your-alb-logs-directory/AWSLogs/XXXXXXXXX/elasticloadbalancing/eu-central-1)

This query ran against the "default" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: XXXXXXX

这是我要运行的查询：

SELECT COUNT(request_verb) AS count,
         request_verb,
         client_ip
FROM alb_logs
GROUP BY  request_verb, client_ip LIMIT 100

任何想法如何解决这个问题？

S3 存储桶存储在同一个根账户上。

我尝试将以下策略添加到 Athena s3 查询存储桶：

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*",
                "arn:aws:s3:::my-bucket"
            ]
        }
    ]
}

【问题讨论】：

可能允许的操作不正确。您要执行的 sdk 操作是什么？看看这里docs.aws.amazon.com/athena/latest/ug/udf-iam-access.html
我想说的是您允许 ListBucket、PutObject 和 GetObject。也许您正在尝试其他操作，例如 Query。我这么说不是因为我与 Athena 合作，而是与 DynamoDB 或 RDS ant 合作，这是主要原因

标签： amazon-web-services amazon-s3

【解决方案1】：

您是否允许您的 ALB 日志 s3 存储桶访问表单服务？

我在 ALB 日志上有一个 s3 资源策略，其中包括

{
"Version": "2012-10-17",
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::AWSACCOUNT:root"
    },
    "Action": "s3:PutObject",
    "Resource": [
      "arn:aws:s3:::BUCKETNAME/PATH/AWSLogs/AWSACCOUNT/*"
    ]
  },
  {
    "Effect": "Allow",
    "Principal": {
      "Service": "delivery.logs.amazonaws.com"
    },
    "Action": "s3:PutObject",
    "Resource": [
      "arn:aws:s3:::BUCKETNAME/PATH/AWSLogs/ACCOUNT_NUMBER/*"
    ],
    "Condition": {
      "StringEquals": {
        "s3:x-amz-acl": "bucket-owner-full-control"
      }
    }
  },
  {
    "Effect": "Allow",
    "Principal": {
      "Service": "delivery.logs.amazonaws.com"
    },
    "Action": "s3:GetBucketAcl",
    "Resource": "arn:aws:s3:::BUCKETNAME"
  }
]
}

【讨论】：