v2 Transfer Manager preview throws NullPointerException

[danbills]

Describe the bug

When downloading a file, we are seeing NullPointerExceptions thrown:a

2022-08-30 18:25:53.781	Exception in thread "Thread-2" java.lang.NullPointerException
2022-08-30 18:25:53.781		at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
2022-08-30 18:25:53.781		at software.amazon.awssdk.crt.s3.S3MetaRequestResponseHandlerNativeAdapter.onResponseBody(S3MetaRequestResponseHandlerNativeAdapter.java:15)

Expected Behavior

We did not expect Null Pointer Exceptions.

Current Behavior

Downloading a file throws an NPE

Reproduction Steps

Unable to consistently reproduce. Seeing it on larger files being downloaded, like ~ 2 GB.

Possible Solution

S3 response body may be empty?

Additional Information/Context

No response

AWS Java SDK version used

2.17.264-PREVIEW

JDK version used

8

Operating System and version

Linux

[danbills]

Also shoul add we were setting targetBandwidth and partSize to:

2Gbps and 256MB, respectively

[yasminetalby]

Hello @danbills ,

Thank you very much for your submission.
I need more information to investigate this issue.

1. Could you please share a reproducible code sample.
2. Could you share the CRT Trace logs to help investigating the origin of this behavior? You can enable CRT Trace logs by adding the following to your code:

Log.initLoggingToFile(Log.LogLevel.Trace, <filename>);

Note : make sure to redact any sensitive information before sharing the file here.

Best,

Yasmine

[danbills]

Hi @yasminetalby , thanks very much for your prompt attention to this!

Could you please share a reproducible code sample.

I'm afraid we are unable to reproduce this consistently, just noticing a trend of it occurring on larger files. The responses from S3 seem to be where it's happening (onResponseBody) so my guess is we would have to mock some kind of unusual S3 response to reproduce.

Could you share the CRT Trace logs to help investigating the origin of this behavior?

Sure, will follow up with that info.

[yasminetalby]

Hello @danbills ,

The CRT logs will allow for further investigation of the behavior.
Thank you for your collaboration!

Best,

Yasmine

[danbills]

tx_manager_bug.txt

Error is at 13:41:58.032

best viewed with less -r as it is ANSI-encoded.

@yasminetalby

[yasminetalby]

Hello @danbills ,

Thank you very much for the documentation provided and for bringing this issue to our attention.
I was wondering if you could give me more context on your use of the S3 transfer manager?
Could you provide details on your client configuration? What credential provider when attempting the download?

Thank you very much for your collaboration!
Sincerely,

Yasmine

[yasminetalby]

Hello @danbills ,

Update: After investigation, I have reached out to the CRT team as this issue seems to be originated from the CRT client which is used by the S3TransferManager.
They have confirmed that this is a known issue on their end and that it will be resolved before GA. (Note that both the S3TransferManager and the CRT Client are currently in PREVIEW)

The current mitigation for this behavior is to increase heap size.
Let me know if you have any further questions or concern.

Thank you very much for submitting this issue and for your collaboration.

Best regards,

Yasmine