ESQL: Safeguards against huge transport requests

[alex-spies]

Executing ESQL queries normally requires that the coordinator node and data nodes communicate: the coordinator sends logical plans to the data nodes, the data nodes send pages with results back to the coordinator.

In both directions, the transport message size seems to be unbounded, and there also seems to be no circuit breaker; we've seen cases where particularly large logical plans caused gigabytes of data to be in buffered in the NettyAllocator.

While some issues were addressed in #112008, #111447 and #111973, we should find other situations where this can happen, test it and fix it if needed. I.e.

• Try to provoke huge LogicalPlans in other ways and test this.
• Try to provoke huge pages to be sent from data nodes, e.g. super many columns, or with individual values that are huge etc.

This is similar to our HeapAttack tests, but distributed.

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[alex-spies]

This relates to a general problem where transport messages can become arbitrarily large; we should at least double check if this should maybe be solved in Elasticsearch more generally, rather than specifically for ESQL.

[wchaparro]

Closing as won't do.