使用 dynamodb 增强客户端时获取 Crc32MismatchException

Question

描述错误

我们尝试使用 DynamoDbEnhancedClient 获取 Item，但我们得到了 Crc32MismatchException。

预期行为

我应该能够得到Item

当前行为

得到错误：

software.amazon.awssdk.core.exception.Crc32MismatchException：预期 1657156166 作为 Crc32 校验和，但实际计算的校验和 是 3693931191 在 software.amazon.awssdk.core.exception.Crc32MismatchException$BuilderImpl.build(Crc32MismatchException.java:88) 在 software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.validateChecksum(Crc32ChecksumValidatingInputStream.java:62) 在 software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.close(Crc32ChecksumValidatingInputStream.java:50) 在 java.base/java.io.FilterInputStream.close（未知来源） software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:124) 在 software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:140) 在 software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.lambda$handle$4(JsonResponseHandler.java:94) 在 java.base/java.util.Optional.ifPresent（未知来源） software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:94) 在 software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36) 在 software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:44) 在 software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52) 在 software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:64) 在 software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52) 在 software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:94) 在 software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$resultTransformationResponseHandler$7(BaseClientHandler.java:287) 在 software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleSuccessResponse(CombinedResponseHandler.java:97) 在 software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:72) 在 software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59) 在 software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) 在 software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:80) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) 在 software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) 在 software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) 在 software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31) 在 software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) 在 software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute（ExecutionFailureExceptionReportingStage.java:37） 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute（ExecutionFailureExceptionReportingStage.java:26） 在 software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:135) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:161) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:114) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:169) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:95) 在 software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45) 在 software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55) 在 software.amazon.awssdk.services.dynamodb.DefaultDynamoDbClient.getItem(DefaultDynamoDbClient.java:3107) 在 software.amazon.awssdk.enhanced.dynamodb.internal.operations.CommonOperation.execute(CommonOperation.java:114) 在 software.amazon.awssdk.enhanced.dynamodb.internal.operations.TableOperation.executeOnPrimaryIndex(TableOperation.java:59) 在 software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:139) 在 software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:146) 在 software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:151)

复制步骤

// https://www.http4k.org/api/org.http4k.client/-ok-http/
val httpClient: HttpHandler = OkHttp()
val awsHttpClient = AwsSdkClient(httpClient)
val dynamoDbClient: DynamoDbClient =
    DynamoDbClient.builder()
        .region(Region.of(environment.getAwsRegion()))
        .httpClient(awsHttpClient)
        .build()

val enhancedClient: DynamoDbEnhancedClient =
    DynamoDbEnhancedClient.builder()
        .dynamoDbClient(dynamoDbClient)
        .build()
val tableName = ...
val schema = ...
val table = enhancedClient.table(tableName, schema)
val key = ..
table.getItem(key)

可能的解决方案

这在 aws-sdk-java 中已经发生，也许它在 v2 中被遗漏了？ aws/aws-sdk-java#1018

使用的 AWS Java SDK 版本

2.17.27

使用的 JDK 版本

Java 11

操作系统和版本

亚马逊 Linux 2

举报给github/aws/aws-sdk-java-v2

Related but not the same stackoverflow question

Answer 1

我以前没有看到其他人为他们的 AWS 客户端使用 OkHttp，我相信这个 HTTP 客户端默认使用 gzip 压缩。要更改配置，您需要按照here 的描述编写一个网络拦截器。

我建议首先使用默认的 HTTP 客户端进行构建，这只需要您删除 2 行代码。这将使您了解 OkHTTP 是否确实是问题所在。

Answer 2

如果您使用 Okhttp 作为客户端，您可以添加自定义标头配置：Accept-Encoding: gzip

你可以在科特林中做这样的事情：

val httpClient: HttpHandler = OkHttp()
val awsHttpClient = AwsSdkClient(httpClient)
val dynamoDbClient: DynamoDbClient =
    DynamoDbClient.builder()
        .region(Region.of(environment.getAwsRegion()))
        .httpClient(awsHttpClient)
        .overrideConfiguration { o -> o.putHeader("Accept-Encoding", "gzip") }
        .build()

val enhancedClient: DynamoDbEnhancedClient =
    DynamoDbEnhancedClient.builder()
        .dynamoDbClient(dynamoDbClient)
        .build()
val tableName = ...
val schema = ...
val table = enhancedClient.table(tableName, schema)
val key = ..
table.getItem(key)