After updating an EC2 app to v3.7.101.61 of AWSSDK.S3, uploading to S3 results in signature error

[bradstiff]

Describe the bug

My ASP.NET core app runs on an EC2 instance. No specific credentials are provided in the configuration.

After updating the AWS SDK from v3 to v4, I get the following error when the app tries to upload an object to S3:

Amazon.S3.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

It should work like it did previously.

Current Behavior

Object upload to S3 fails.

Reproduction Steps

Service configuration in Startup.cs looks like this:

services.AddDefaultAWSOptions(Configuration.GetAWSOptions());
services.AddAWSService<IAmazonS3>();

Configuration section in appsettings.json looks like this:

"AWS": {
    "Region": "us-east-2"
},

Possible Solution

No response

Additional Information/Context

No code was changed.

No permissions were changed.

EC2 Instance role is the same.

AWS .NET SDK and/or Package version used

Previous package versions:

    <PackageReference Include="AWS.Logger.AspNetCore" Version="3.3.0" />
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="3.7.2" />
    <PackageReference Include="AWSSDK.S3" Version="3.7.101.21" />

New package versions (where the error occurs):

    <PackageReference Include="AWS.Logger.AspNetCore" Version="4.0.0" />
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="4.0.0" />
    <PackageReference Include="AWSSDK.S3" Version="4.0.0.4" />

Targeted .NET Platform

.NET 9

Operating System and version

64bit Amazon Linux 2023 v3.4.1 running .NET 9

[ashishdhingra]

@bradstiff Good afternoon. Thanks for opening the issue. Could you please share minimal reproducible project to troubleshoot the issue?

In V4, we always default to SigV4 for signing requests (refer #3362). There was also a campaign to deprecate SigV2 for new buckets.

Are you by any by any chance:

• Using SigV2?
  • If yes, using new S3 bucket which disallows SigV2 per above campaign?
• Have AWS Options configured in appsettings.json file?
• IAM policy that restricts to use specific signature version.

Thanks,
Ashish

[bradstiff]

Since this is most likely caused by AWS configuration details and/or a bug within the SDK, I don't think a code example is practical.

I don't know if I am using SigV2 or not.

The bucket has existed for five years.

I have already provided the full code that configures the IAmazonS3 service in the DI container, including the relevant options in the appsettings.json file.

No signature version policy restrictions exist.

The code where the error occurs looks like this, where _s3Client is an instance of IAmazonS3 that is injected by the DI container:

            var fileTransferUtility = new TransferUtility(_s3Client);
            using (var stream = new MemoryStream())
            {
                await fileTransferUtility.UploadAsync(stream, _photoBucket, $"{id}.jpg"); <---FAILS HERE
            }

[bradstiff]

I found this is not a v4 issue.

I tried to narrow down the version of AWSSDK.S3 where it starts breaking.

3.7.10.1 works
3.7.101.61 (not a typo) does not work

There are over 60 versions between those two versions; I did not have time to be more specific.

[dscpinheiro]

Does _photoBucket include part of the key prefix? (e.g. "mybucket/some-path")

Looking at the version this started happening it seems like it's related to this change introduced in 3.7.100: #2622 (comment)