无法将 Kinesis 数据流发送到 DynamoDB答案

【问题标题】：Can not send a Kinesis data stream to DynamoDB无法将 Kinesis 数据流发送到 DynamoDB
【发布时间】：2021-03-31 14:47:36
【问题描述】：

我在将数据流从 kinesis 发送到 DynamoDB 时遇到问题。我使用这个 lambda 函数来完成这项工作。当我尝试 lambda 中的测试按钮时，它会将数据发送到 DynamoDB。但是，当我尝试从数据流发送时，我收到此错误“字符串索引必须是整数”。请问有什么解决办法吗？谢谢

这里是代码

import boto3
import datetime
import base64
import json

def lambda_handler(event, context):
    try:
        dynamo_db = boto3.resource('dynamodb')
        table = dynamo_db.Table('detected-faces')
        for record in event["Records"]:
            encoded = record["kinesis"]["data"]
            decoded = json.loads(base64.b64decode(encoded).decode("utf-8"))
            for detectedFaces in decoded:
                matchedFaces = detectedFaces["MatchedFaces"]
                for face in matchedFaces:
                    data = {
                        'FaceId': face["Face"]["FaceId"],
                        'Confidence': str(face["Face"]["Confidence"]),
                        'Similarity': str(face["Similarity"]),
                        'ExternalImageId':face["Face"]["ExternalImageId"],
                        'ImageId':face["Face"]["ImageId"]
                    }
                    # table.put_item(Item=data)  
                    with table.batch_writer() as batch_writer:
                        batch_writer.put_item(Item=data)
    except Exception as e: 
        print(str(e))

这里是 lambda 测试

{
  "Records": [
    {
      "kinesis": {
        "kinesisSchemaVersion": "1.0",
        "partitionKey": "4522972768",
        "sequenceNumber": "49616873388568555882989759900514551623368944523335958530",
        "data": "W3siRGV0ZWN0ZWRGYMDYyMzUzNS1kYWY4LTQ5MjgtYmM2ZS1hOTk0MzliMTIzYmEiLCJDb25maWRlbmNlIjo5OS45NjczLCJJbWFnZUlkIjoiYjhjMDYyNWItOGEwMS0zNGJmLWJkZGMtMGM4YmI5NGMyMGFmIiwiRXh0ZXJuYWxJbWFnZUlkIjoiYWxhYSJ9fV19XQo=",
        "approximateArrivalTimestamp": 1617193796.445
      },
      "eventSource": "aws:kinesis",
      "eventVersion": "1.0",
      "eventID": "shardId-000000000000:496168733885685558829897599005148787423368944523335958530",
      "eventName": "aws:kinesis:record",
      "invokeIdentityArn": "arn:aws:iam::731558896689:role/lambda-kinesis-dynamodb",
      "awsRegion": "eu-west-1",
      "eventSourceARN": "arn:aws:kinesis:eu-west-1:731558896689:stream/dynmoDB-kds-stream"
    }
  ]
}

这里是数据流的返回：

START RequestId: 1f40c1db-7f67-4fd2-8f2e-637c94ab08b8 Version: $LATEST
string indices must be integers
END RequestId: 1f40c1db-7f67-4fd2-8f2e-637c94ab08b8
REPORT RequestId: 1f40c1db-7f67-4fd2-8f2e-637c94ab08b8  Duration: 56.43 ms  Billed Duration: 57 ms  Memory Size: 128 MB Max Memory Used: 72 MB  
START RequestId: 0742e744-1b85-4169-bdcf-b6e0f902d9c4 Version: $LATEST
string indices must be integers
END RequestId: 0742e744-1b85-4169-bdcf-b6e0f902d9c4
REPORT RequestId: 0742e744-1b85-4169-bdcf-b6e0f902d9c4  Duration: 48.96 ms  Billed Duration: 49 ms  Memory Size: 128 MB Max Memory Used: 72 MB  
START RequestId: c3414096-d77d-4db2-ab1f-7ab48c58c627 Version: $LATEST
string indices must be integers
END RequestId: c3414096-d77d-4db2-ab1f-7ab48c58c627
REPORT RequestId: c3414096-d77d-4db2-ab1f-7ab48c58c627  Duration: 105.43 ms Billed Duration: 106 ms Memory Size: 128 MB Max Memory Used: 72 MB

【问题讨论】：

只需打印 event 并检查实际输入的内容。一般建议是，batch_writer 应该在所有 for 循环周围打开，而不是在它们内部。
感谢您的回复。如上所示，它正在打印“记录”。解码 base64 时，我得到如下信息： [{"DetectedFace":{"BoundingBox":{"Height":0.3523918,"Width":0.19285616,"Left":0.3260747,"Top":0.4626954},"Confidence" :99.99663,"MatchedFaces":{"Similarity":99.688065,"Face":{"BoundingBox":{"Height":0.623835,"Width":0.202785,"Left":0.80337,"Top":0.192938}," FaceId":"50623535-daf8-4928-bc6e-a99439b123ba","信心":99.9673,"ImageId":"b8c0625b-8a01-34bf-bddc-0c8bb94c20af","ExternalImageId":"john"}}]}]
如果你手动解码base64编码的数据会发生什么，所有的记录都符合预期的格式吗？能否显示调用失败的记录？
是的，所有记录都符合预期的格式。在 lambda 函数中进行测试时，一切顺利。它将数据存储在数据库中。启动流时，lambda函数的日志是这样的：``` START RequestId: 1f40c1db-7f67-4fd2-8f2e-637c94ab08b8 Version: $LATEST string indices must be integers END RequestId: 1f40c1db-7f67-4fd2-8f2e-637c94ab08b8报告请求 ID：1f40c1db-7f67-4fd2-8f2e-637c94ab08b8 持续时间：56.43 毫秒计费持续时间：57 毫秒内存大小：128 MB 使用的最大内存：72 MB ```
我猜它在某处缺少一个 for 循环，但我想不通

标签： python amazon-web-services aws-lambda amazon-dynamodb amazon-kinesis

【解决方案1】：

已解决 for循环应该是这样的

for record in event["Records"]:
            encoded = record["kinesis"]["data"]
            decoded = json.loads(base64.b64decode(encoded).decode("utf-8"))
            for faceSearchResponse in decoded["FaceSearchResponse"]:
                for matchedFaces in faceSearchResponse["MatchedFaces"]:

【讨论】：