[CI] FullClusterRestartSystemIndexCompatibilityIT testAsyncSearchIndexMigration {p0=9.0.1} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #7864 / rhel-8_platform-support-unix
• elasticsearch-periodic-platform-support #7828 / almalinux-8-aarch64_checkpart1_platform-support-arm
• elasticsearch-periodic-platform-support #7786 / windows-2022_checkpart1_platform-support-windows

Reproduction Line:

./gradlew ":qa:lucene-index-compatibility:javaRestTest" --tests "org.elasticsearch.lucene.FullClusterRestartSystemIndexCompatibilityIT" -Dtests.method="testAsyncSearchIndexMigration {p0=9.0.1}" -Dtests.seed=D0E2A7E45CFFF83D -Dtests.locale=dz-Tibt-BT -Dtests.timezone=America/Santo_Domingo -Druntime.java=24

Applicable branches:
9.0

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.RuntimeException: An error occurred orchestrating test cluster.

Issue Reasons:

• [9.0] 3 failures in test testAsyncSearchIndexMigration {p0=9.0.1} (0.8% fail rate in 354 executions)
• [9.0] 3 failures in pipeline elasticsearch-periodic-platform-support (21.4% fail rate in 14 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[cbuescher]

This looks like occasionally the read-only setting for the index that this test is migrating isn't updated correctly before the cluster upgrade. I see the following in the logs:

[2025-04-25T09:24:19,054][ERROR][o.e.b.Elasticsearch      ] [test-cluster-2] fatal exception while booting Elasticsearch java.lang.IllegalStateException: The index [index-testasyncsearchindexmigration/ZGjTIzENTBuZFT-FlTPyiw] created in version [7.17.25-8.0.0] with current compatibility version [7.17.25-8.0.0] must be marked as read-only using the setting [index.blocks.write] set to [true] before upgrading to 9.0.1.

I assume that this isn't a general problem with applying the setting because this works almost always, also in manual tests, but rather an infrequent case in the cluster and test orchestration. Marking as "low risk" for that reason.

[cbuescher]

occasionally the read-only setting for the index that this test is migrating isn't updated correctly before the cluster upgrade

I have to correct this assessment. The settings update error messages always appear only in the "testAsyncSearchIndexMigration {p0=9.0.1}" test run where the previous testAsyncSearchIndexMigration {p0=8.18.0} also is failing. So I assume that the update from version 7 to version 8 cluster already doesn't work here (see #127245) and the 9 upgrade only fails because the previous test step didn't run successfully.
I'll close this issue as a duplicate (or result) of #127245 for that reason.