[CI] SSLErrorMessageFileTests class failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #6650 / openjdk24_entitlements_true_checkpart4_java-matrix
• elasticsearch-periodic #6650 / openjdk24_entitlements_false_checkpart4_java-matrix
• elasticsearch-periodic #6643 / openjdk24_entitlements_false_checkpart4_java-matrix
• elasticsearch-periodic #6643 / openjdk24_entitlements_true_checkpart4_java-matrix
• elasticsearch-periodic #6636 / openjdk24_entitlements_true_checkpart4_java-matrix
• elasticsearch-periodic #6636 / openjdk24_entitlements_false_checkpart4_java-matrix
• elasticsearch-periodic #6629 / openjdk24_entitlements_false_checkpart4_java-matrix
• elasticsearch-periodic #6629 / openjdk24_entitlements_true_checkpart4_java-matrix
• elasticsearch-periodic #6622 / openjdk24_entitlements_false_checkpart4_java-matrix
• elasticsearch-periodic #6622 / openjdk24_entitlements_true_checkpart4_java-matrix

Reproduction Line:

./gradlew ":x-pack:plugin:security:test" --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageFileTests.testMessageForCertificateAuthoritiesOutsideConfigDir" -Dtests.seed=980B56688AF2245A -Dtests.jvm.argline="-Des.entitlements.enabled=true" -Dtests.locale=zgh-Tfng-MA -Dtests.timezone=Asia/Bahrain -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: a throwable with message of "failed to load SSL configuration [xpack.monitoring.exporters.http.ssl] - cannot read configured PEM certificate_authorities [/this/path/is/outside/the/config/directory/file.error] because access to read the file is blocked; SSL resources should be placed in the [/dev/shm/bk/bk-agent-prod-gcp-1740575026537098452/elastic/elasticsearch-periodic/x-pack/plugin/security/build/testrun/test/temp/org.elasticsearch.xpack.ssl.SSLErrorMessageFileTests_980B56688AF2245A-001/tempDir-053/config] directory"
     but: was <org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.monitoring.exporters.http.ssl] - cannot read configured PEM certificate_authorities [/this/path/is/outside/the/config/directory/file.error] because the file does not exist> at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:622)

Issue Reasons:

• [main] 17 consecutive failures in step openjdk24_entitlements_true_checkpart4_java-matrix
• [main] 17 consecutive failures in step openjdk24_entitlements_false_checkpart4_java-matrix
• [main] 34 failures in class org.elasticsearch.xpack.ssl.SSLErrorMessageFileTests (3.5% fail rate in 958 executions)
• [main] 17 failures in step openjdk24_entitlements_true_checkpart4_java-matrix (100.0% fail rate in 17 executions)
• [main] 17 failures in step openjdk24_entitlements_false_checkpart4_java-matrix (100.0% fail rate in 17 executions)
• [main] 17 failures in pipeline elasticsearch-periodic (100.0% fail rate in 17 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-security (Team:Security)

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[n1v0lg]

This passes reliably without entitlements and fails with them enabled:

./gradlew ":x-pack:plugin:security:test" --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageFileTests.testMessageForPemKeyOutsideConfigDir" -Dtests.jvm.argline="-Des.entitlements.enabled=true" -Dtests.locale=prg-PL -Dtests.timezone=Etc/GMT+1 -Druntime.java=24

(Note that no seed is needed above).

Based on the assertion failure, I'm assuming that we're missing entitlements-related config:

Expected:

failed to load SSL configuration [xpack.security.transport.ssl] - cannot read configured PEM private key [/this/path/is/outside/the/config/directory/file.error] because access to read the file is blocked; SSL resources should be placed in the [<omitted>/x-pack/plugin/security/build/testrun/test/temp/org.elasticsearch.xpack.ssl.SSLErrorMessageFileTests_42EC0C234925F23B-001/tempDir-002/config] directory

But was:

failed to load SSL configuration [xpack.security.transport.ssl] - cannot read configured PEM private key [/this/path/is/outside/the/config/directory/file.error] because the file does not exist

Could someone from Core Infra take a look? Thanks!

[rjernst]

This looks like a test-only problem. We need test config for entitlements to fix.
cc @prdoyle

[rjernst]

@n1v0lg This is not entitlements related (note that although entitlements are in the jvm args, that is only used when running ES currently, not in unit tests such as this one). The problem is this test ran on JDK 24, where no security manager can be used. It looks like the test (or the ssl error message code) is specifically looking for security manager exceptions. It should be refactored to work without that.

[n1v0lg]

Good catch @rjernst -- on my plate to take another look 👀

[mosche]

@n1v0lg any progress on this? This blocks us from migrating to JDK 24

[n1v0lg]

@mosche I'll take a look today or find someone else from the security team.

[mosche]

Thanks @n1v0lg, I had a brief look. The failing tests rely on security manager denying the operation, this won't be possible using entitlements (at last at the moment). Entitlement checks are not enabled for unit tests, but only when running Elasticsearch (e.g. in REST tests).
I'll raise a PR to skip these tests if running without security manager.

[mosche]

Fixing this issue will require a follow up by @elastic/es-security to migrate code from handling AccessControlException to NotEntitledException

Respective tests would have to be implemented as REST tests