在 AWS Lambda 上运行 python 脚本时出错答案

【问题标题】：Error while running a python script on AWS Lambda在 AWS Lambda 上运行 python 脚本时出错
【发布时间】：2020-12-11 04:43:48
【问题描述】：

我正在尝试在 AWS Lambda 上运行以下 python 脚本，该脚本是我手动运行的，我可以在输出 S3 存储桶上获得结果而没有任何问题。但是现在当我从 AWS Lambda 调用脚本时出现以下错误，不确定我是否遗漏了脚本上的任何内容？

#!/usr/bin/env python3
import boto3

#Function for executing athena queries
def run_query(Event, context):
    ...
    run_query(query, database, s3_output)
    client = boto3.client('athena')
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': 's3_accesslog'
            },
        ResultConfiguration={
            'OutputLocation': s3_output,
            }
        )
    
#import datetime 
import datetime
year = datetime.date.today()
year = year.strftime("%Y")
month = datetime.date.today()
month = month.strftime("%m")
day = datetime.date.today()
day = day.strftime("%d")

#select bucket 
s3_input = "s3://smathena/cf-ant-prod/year=%s/month=%s/day=%s" % (year, month, day)
   
#Athena configuration
s3_ouput = 's3://smathena/athenatest/'
database = 's3_accesslog'
table = 'test_output1'

#Athena database and table definition
create_database = "CREATE DATABASE IF NOT EXISTS %s;" % (database)
delete_table = "drop table %s.%s;" % (database, table)
create_table = \
  """CREATE EXTERNAL TABLE IF NOT EXISTS %s.%s (
  `Date` DATE,
   Time STRING,
   Location STRING,
   SCBytes BIGINT,
   RequestIP STRING,
   Method STRING,
   Host STRING,
   Uri STRING,
   Status INT,
   Referrer STRING,
   UserAgent STRING,
   UriQS STRING,
   Cookie STRING,
   ResultType STRING,
   RequestId STRING,
   HostHeader STRING,
   Protocol STRING,
   CSBytes BIGINT,
   TimeTaken FLOAT,
   XForwardFor STRING,
   SSLProtocol STRING,
   SSLCipher STRING,
   ResponseResultType STRING,
   CSProtocolVersion STRING,
   FleStatus STRING,
   FleEncryptedFields INT,
   CPort INT,
   TimeToFirstByte FLOAT,
   XEdgeDetailedResult STRING,
   ScContent STRING,
   ScContentLen BIGINT,
   ScRangeStart BIGINT,
   ScRangeEnd BIGINT
   )
   ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
   LOCATION '%s'
   TBLPROPERTIES ('skip.header.line.count' = '2');""" % (database, table, s3_input)

#Query definitions
query_1 = "SELECT * FROM %s.%s where CAST(status AS VARCHAR) = '404';" % (database, table)

#Execute all queries
queries = [ create_database, delete_table, create_table, query_1 ]
for q in queries:
   print("Executing query: %s" % (q))
   res = run_query(q, database, s3_ouput)

但是现在当我从 AWS Lambda 调用脚本时出现以下错误，不确定我是否遗漏了脚本上的任何内容？

{
  "errorMessage": "run_query() takes 2 positional arguments but 3 were given",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"<frozen importlib._bootstrap>\", line 696, in _load\n",
    "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 86, in <module>\n    res = run_query(q, database, s3_ouput)\n"
  ]
}``

【问题讨论】：

这个 Lambda 函数是如何被调用的？为query、database 和s3_output 赋予值意味着什么？

标签： python amazon-web-services amazon-s3 aws-lambda amazon-athena

【解决方案1】：

您的函数lambda_handle 不符合python lambda interface：

def handler_name(event, context): 
    ...
    return some_value

函数的输入应该在event 中。该链接的另一个示例：

def my_handler(event, context):
    message = 'Hello {} {}!'.format(event['first_name'], 
                                    event['last_name'])  
    return { 
        'message' : message
    }

在您的情况下，我希望 query、database 和 s3_output 成为 event 的一部分。您可能应该return 有关正在执行的雅典娜查询的信息。

【讨论】：