Skip to content

[CI] S3RepositoryAnalysisRestIT testRepositoryAnalysis failing #127436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
elasticsearchmachine opened this issue Apr 26, 2025 · 2 comments · Fixed by #127442
Closed

[CI] S3RepositoryAnalysisRestIT testRepositoryAnalysis failing #127436

elasticsearchmachine opened this issue Apr 26, 2025 · 2 comments · Fixed by #127442
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Distributed Coordination Meta label for Distributed Coordination team >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Apr 26, 2025 •
edited
Loading

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:snapshot-repo-test-kit:qa:s3:javaRestTest" --tests "org.elasticsearch.repositories.blobstore.testkit.analyze.S3RepositoryAnalysisRestIT.testRepositoryAnalysis" -Dtests.seed=51167EC8B4949CB9 -Dtests.jvm.argline="-Des.entitlements.enabled=false" -Dtests.locale=en-UM -Dtests.timezone=Africa/Cairo -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

org.elasticsearch.client.ResponseException: method [POST], host [http://[::1]:44081], URI [/_snapshot/repository/_analyze?blob_count=10&seed=-7758166901472037605&max_blob_size=10mb&timeout=120s&concurrency=4], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[repository] failed to copy blob after write: [test-blob-4-uvouJIsDR_KGh-ZLU5xSOQ]"}],"type":"repository_verification_exception","reason":"[repository] Elasticsearch observed the storage system underneath this repository behaved incorrectly which indicates it is not suitable for use with Elasticsearch snapshots. Typically this happens when using storage other than AWS S3 which incorrectly claims to be S3-compatible. If so, please report this incompatibility to your storage supplier. Do not report Elasticsearch issues involving storage systems which claim to be S3-compatible unless you can demonstrate that the same issue exists when using a genuine AWS S3 
[truncated]

Issue Reasons:

  • [main] 5 failures in test testRepositoryAnalysis (2.3% fail rate in 217 executions)
  • [main] 2 failures in pipeline elasticsearch-periodic-platform-support (40.0% fail rate in 5 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI labels Apr 26, 2025
elasticsearchmachine added a commit that referenced this issue Apr 26, 2025
@elasticsearchmachine
Mute org.elasticsearch.repositories.blobstore.testkit.analyze.S3Repos…
3057168 
…itoryAnalysisRestIT testRepositoryAnalysis #127436
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 4 failures in test testRepositoryAnalysis (1.9% fail rate in 212 executions)
  • [main] 2 failures in pipeline elasticsearch-periodic-platform-support (40.0% fail rate in 5 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Distributed Coordination Meta label for Distributed Coordination team labels Apr 26, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Apr 27, 2025
@DaveCTurner
Fix UploadPartCopy source handling
7ea50ef 
The leading `/` on the `x-amz-copy-source` header is optional; SDKv1
includes it but SDKv2 does not. Today we handle its omission when
handling a `CopyObject` request but we must do the same for the
`UploadPartCopy` path.

Closes elastic#127436
@DaveCTurner DaveCTurner mentioned this issue Apr 27, 2025
Merged
nicktindall pushed a commit that referenced this issue Apr 28, 2025
@DaveCTurner
Fix UploadPartCopy source handling (#127442)
9772b5e 
The leading `/` on the `x-amz-copy-source` header is optional; SDKv1
includes it but SDKv2 does not. Today we handle its omission when
handling a `CopyObject` request but we must do the same for the
`UploadPartCopy` path.

Closes #127436
benchaplin pushed a commit to benchaplin/elasticsearch that referenced this issue Apr 28, 2025
@elasticsearchmachine @benchaplin
Mute org.elasticsearch.repositories.blobstore.testkit.analyze.S3Repos…
8e85664 
…itoryAnalysisRestIT testRepositoryAnalysis elastic#127436
benchaplin pushed a commit to benchaplin/elasticsearch that referenced this issue Apr 28, 2025
@DaveCTurner @benchaplin
Fix UploadPartCopy source handling (elastic#127442)
ce4cfaa 
The leading `/` on the `x-amz-copy-source` header is optional; SDKv1
includes it but SDKv2 does not. Today we handle its omission when
handling a `CopyObject` request but we must do the same for the
`UploadPartCopy` path.

Closes elastic#127436
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Distributed Coordination Meta label for Distributed Coordination team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix UploadPartCopy source handling DaveCTurner/elasticsearch
1 participant
@elasticsearchmachine