[CI] GetSnapshotsIT testFilterByState failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #8853 / openjdk24_checkpart1_java-matrix
• elasticsearch-pull-request #76620 / part-1
• elasticsearch-pull-request #76154 / part-1

Reproduction Line:

./gradlew ":server:internalClusterTest" --tests "org.elasticsearch.snapshots.GetSnapshotsIT.testFilterByState" -Dtests.seed=9FA772A31CBFDDA3 -Dtests.locale=en-FI -Dtests.timezone=BST -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: null

Issue Reasons:

• [main] 3 failures in test testFilterByState (0.8% fail rate in 382 executions)
• [main] 2 failures in step part-1 (1.5% fail rate in 137 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.4% fail rate in 143 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 3 failures in test testFilterByState (0.8% fail rate in 382 executions)
• [main] 2 failures in step part-1 (1.5% fail rate in 137 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.4% fail rate in 143 executions)

Build Scans:

• elasticsearch-periodic #8853 / openjdk24_checkpart1_java-matrix
• elasticsearch-pull-request #76620 / part-1
• elasticsearch-pull-request #76154 / part-1

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[nicktindall]

This is just a test issue, for whatever reason the blocking of the data nodes doesn't prevent snapshot-in-progress from completing. If you put a safeSleep(5_000); on line 668 you can reproduce this reliably.

cc @JeremyDahlgren