[CI] GenerativeIT test failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #7835 / ubuntu-2404_platform-support-unix
• elasticsearch-pull-request #69189 / part-3
• elasticsearch-pull-request #69186 / part-3
• elasticsearch-pull-request #69156 / part-3
• elasticsearch-pull-request #69087 / part-3
• elasticsearch-pull-request #69079 / part-3
• elasticsearch-periodic-platform-support #7829 / ubuntu-2404-aarch64_checkpart3_platform-support-arm
• elasticsearch-periodic #7789 / encryption-at-rest
• elasticsearch-periodic-platform-support #7811 / rocky-8_platform-support-unix
• elasticsearch-pull-request #68588 / part-3

Reproduction Line:

./gradlew ":x-pack:plugin:esql:qa:server:single-node:javaRestTest" --tests "org.elasticsearch.xpack.esql.qa.single_node.GenerativeIT.test" -Dtests.seed=61E6C94C946D0B3A -Dtests.locale=es-US -Dtests.timezone=Greenwich -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: query: from languages_lookup,message_types | dissect type "%{language_name} %{lUExNPlCFv}" | keep `language_name`, `message`, `type`, `language_code` | drop messag*, language_name | where  NOT false AND true | stats  dZHmcwpytj = min(language_code), language_code = count(*) by type | limit 3173 | enrich languages_policy on type | keep language_code, dZHmcwpytj| rename dZHmcwpytj as language_code | lookup join languages_lookup on language_code | dissect language_name "%{rzEYPygMrvRn} %{VoxwLDJkW}" | dissect rzEYPygMrvRn "%{rzEYPygMrvRn} %{oRDpAlpSPJrS}" | eval  `oRDpAlpSPJrS` = 4069359665708631012, NRIVvYrpf = null, eusQAcRMXvpY = null, pKCqMKIRHa = null, `rzEYPygMrvRn` = true, lnZPOkBpIVRS = -342457484, rzEYPygMrvRn = 1213716962 | dissect VoxwLDJkW "%{CqJzWvklnYU} %{kpENLsKnqNC}" | rename `rzEYPygMrvRn` AS `sdxbyBdM`, `language_name` AS CqJzWvklnYU, NRIVvYrpf AS `mqMgkAlp` | keep `eusQAcRMXvpY`, CqJzWvklnYU, `oRDpAlpSPJrS`, sdxbyBdM, eusQAcRMXvpY, `kpENLsKnqNC
[truncated]

Issue Reasons:

• [main] 2 consecutive failures in test test
• [main] 11 failures in test test (1.2% fail rate in 881 executions)
• [main] 7 failures in step part-3 (2.3% fail rate in 298 executions)
• [main] 3 failures in pipeline elasticsearch-periodic-platform-support (16.7% fail rate in 18 executions)
• [main] 7 failures in pipeline elasticsearch-pull-request (2.4% fail rate in 295 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 3 failures in test test (0.4% fail rate in 692 executions)
• [main] 2 failures in step part-3 (1.0% fail rate in 203 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.0% fail rate in 200 executions)

Build Scans:

• elasticsearch-periodic-platform-support #7811 / rocky-8_platform-support-unix
• elasticsearch-pull-request #68588 / part-3
• elasticsearch-pull-request #68457 / part-3

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[alex-spies]

@luigidellaquila , going by the verification exception message, it looks like our generative tests genuinely constructed an invalid query; could you maybe take a look and confirm, resp. exclude such queries to unmute the generative tests?

[alex-spies]

I think this is medium risk due to the whole generative tests being muted.

[luigidellaquila]

Thanks @alex-spies, the error message in the issue is just a problem in the test (easy fix), but I also I see some new errors in the other CI failures, there could be an actual regression.
I'll check it ASAP

[luigidellaquila]

Yes, I can confirm: new bug #127167

[luigidellaquila]

Unmute here #127211

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 2 consecutive failures in test test
• [main] 11 failures in test test (1.2% fail rate in 881 executions)
• [main] 7 failures in step part-3 (2.3% fail rate in 298 executions)
• [main] 3 failures in pipeline elasticsearch-periodic-platform-support (16.7% fail rate in 18 executions)
• [main] 7 failures in pipeline elasticsearch-pull-request (2.4% fail rate in 295 executions)

Build Scans:

• elasticsearch-periodic-platform-support #7835 / ubuntu-2404_platform-support-unix
• elasticsearch-pull-request #69189 / part-3
• elasticsearch-pull-request #69186 / part-3
• elasticsearch-pull-request #69156 / part-3
• elasticsearch-pull-request #69087 / part-3
• elasticsearch-pull-request #69079 / part-3
• elasticsearch-periodic-platform-support #7829 / ubuntu-2404-aarch64_checkpart3_platform-support-arm
• elasticsearch-periodic #7789 / encryption-at-rest
• elasticsearch-periodic-platform-support #7811 / rocky-8_platform-support-unix
• elasticsearch-pull-request #68588 / part-3

[luigidellaquila]

The tests spotted a new bug.
Plus, some checks were not accurate and another bug was not completely fixed apparently.