[CI] SimpleSecurityNetty4ServerTransportTests testClientChannelUsesSeparateSslConfigurationForRemoteCluster failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #9197 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9183 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9176 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9169 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9162 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9155 / openjdk17_checkpart4_java-fips-matrix
• elasticsearch-periodic #9133 / openjdk17_checkpart4_java-fips-matrix

Reproduction Line:

./gradlew ":x-pack:plugin:security:test" --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testClientChannelUsesSeparateSslConfigurationForRemoteCluster" -Dtests.seed=54A39CE6B8BD8061 -Dtests.locale=teo-UG -Dtests.timezone=America/Anchorage -Druntime.java=17 -Dtests.fips.enabled=true

Applicable branches:
8.19

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: (a string containing "unable to find valid certification path" or a string containing "Unable to find certificate chain" or a string containing "Unable to construct a valid chain")
     but: was "Unable to build a CertPath: no PKIXBuilderParameters available"

Issue Reasons:

• [8.19] 7 consecutive failures in step openjdk17_checkpart4_java-fips-matrix
• [8.19] 7 failures in test testClientChannelUsesSeparateSslConfigurationForRemoteCluster (2.2% fail rate in 324 executions)
• [8.19] 7 failures in step openjdk17_checkpart4_java-fips-matrix (100.0% fail rate in 7 executions)
• [8.19] 7 failures in pipeline elasticsearch-periodic (70.0% fail rate in 10 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[ywangd]

It's reproducible and the underlying error is entitlement.

[2025-07-05T11:44:30,396][WARN ][o.e.e.r.p.P.s.ALL-UNNAMED] [testClientChannelUsesSeparateSslConfigurationForRemoteCluster] Not entitled: component [security], module [ALL-UNNAMED], class [class org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi], entitlement [file], operation [read], path [/dev/shm/bk/bk-agent-prod-gcp-1751747817495865680/elastic/elasticsearch-periodic/x-pack/plugin/security/build/fips-resources/cacerts.bcfks]
org.elasticsearch.entitlement.runtime.api.NotEntitledException: component [security], module [ALL-UNNAMED], class [class org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi], entitlement [file], operation [read], path [/dev/shm/bk/bk-agent-prod-gcp-1751747817495865680/elastic/elasticsearch-periodic/x-pack/plugin/security/build/fips-resources/cacerts.bcfks]
	at org.elasticsearch.entitlement.runtime.policy.PolicyCheckerImpl.notEntitled(PolicyCheckerImpl.java:471) ~[elasticsearch-entitlement-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.entitlement.runtime.policy.PolicyCheckerImpl.checkFileRead(PolicyCheckerImpl.java:246) ~[elasticsearch-entitlement-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.entitlement.runtime.policy.PolicyCheckerImpl.checkFileRead(PolicyCheckerImpl.java:209) ~[elasticsearch-entitlement-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.entitlement.runtime.policy.PolicyCheckerImpl.checkFileRead(PolicyCheckerImpl.java:203) ~[elasticsearch-entitlement-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.entitlement.runtime.policy.ElasticsearchEntitlementChecker.check$java_io_File$exists(ElasticsearchEntitlementChecker.java:1422) ~[elasticsearch-entitlement-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at java.io.File.exists(File.java) ~[?:?]
	at org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi.getDefaultTrustStore(ProvTrustManagerFactorySpi.java:55) ~[bctls-fips-1.0.19.jar:1.0.19]
	at org.bouncycastle.jsse.provider.ProvTrustManagerFactorySpi.engineInit(ProvTrustManagerFactorySpi.java:162) ~[bctls-fips-1.0.19.jar:1.0.19]
	at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:282) ~[?:?]
	at org.elasticsearch.common.ssl.KeyStoreUtil.createTrustManager(KeyStoreUtil.java:172) ~[elasticsearch-ssl-config-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.common.ssl.DefaultJdkTrustConfig.createTrustManager(DefaultJdkTrustConfig.java:66) ~[elasticsearch-ssl-config-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
	at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:479) ~[x-pack-core-8.19.0-SNAPSHOT.jar:8.19.0-SNAPSHOT]
...

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)