S3TransferManager upload failure with ChecksumAlgorithm

[chappidim]

Describe the bug

S3TransferManager upload fails when we any checkSumAlgorithm

Expected Behavior

S3TransferManager upload with checkSumAlgorithm

Current Behavior

16 Sep 2022 16:47:34,359 [INFO]  (ShardRecordProcessor-0001) software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener: Transfer initiated...
--
16 Sep 2022 16:47:34,359 [INFO]  (ShardRecordProcessor-0001) software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener: \|                    \| 0.0%
16 Sep 2022 16:47:34,360 [DEBUG]  (ShardRecordProcessor-0001) software.amazon.awssdk.request: Sending Request: DefaultSdkHttpFullRequest(httpMethod=PUT, protocol=https, host=BUCKET_BLUR-us-east-1.s3.amazonaws.com, port=443, encodedPath=/KEY_BLUR, headers=[amz-sdk-invocation-id, Content-Length, Content-Type, Expect, User-Agent, x-amz-sdk-checksum-algorithm], queryParameters=[])
16 Sep 2022 16:47:34,361 [INFO]  (Thread-26) software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener: \|====================\| 100.0%
16 Sep 2022 16:47:34,391 [WARN]  (sdk-async-response-27-10) ...
java.util.concurrent.CompletionException: software.amazon.awssdk.services.s3.model.S3Exception: x-amz-sdk-checksum-algorithm specified, but no corresponding x-amz-checksum-* or x-amz-trailer headers were found. (Service: S3, Status Code: 400, Request ID: GF0Z5M2ZEGCEENQJ, Extended Request ID: sOjG+TZRp1XNRYZazMsXxLOyvJjBTNttwAFsj/+PwPigoXHSw/A8yRjJMXEWbvuGu134nhfx4JXTXha6u2wC7A==)	at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:62)	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:76)	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:103)	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:181)	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:167)	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$null$0(MakeAsyncHttpRequestStage.java:105)	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:163)	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)	at java.base/java.lang.Thread.run(Thread.java:829)

Reproduction Steps

Simple file upload with checkSum enable

      FileUpload upload = s3TransferManager.uploadFile(UploadFileRequest.builder()
          .source(tempFile)
          .putObjectRequest(PutObjectRequest.builder()
              .bucket(config.getBucket())
              .key(config.getKey())
              .checksumAlgorithm(ChecksumAlgorithm.CRC32)
              .build())
          .overrideConfiguration(o -> o.addListener(LoggingTransferListener.create()))
          .build());

Trying different ChecksumAlgorithm (SHA256) and failed with same exception

Possible Solution

NA

Additional Information/Context

No response

AWS Java SDK version used

2.0.3974

JDK version used

OpenJDK_64-Bit_Server_VM/11.0.16+10-LTS java/11.0.16

Operating System and version

Linux/4.14.290-217.505.amzn2.x86_64

[zoewangg]

Flexible checksum support was added via #3376 and #3391. The PRs have been merged but we haven't release the change yet. We will post here once it's released.

[zoewangg]

Hi @chappidm, just let you know that the change has been released. Could you try with the latest version (2.19.0 as of today)