ES|QL: enable EXPLAIN (snapshot only)

[luigidellaquila]

Re-enabling a very basic EXPLAIN command.
It's in snapshot mode only for now, and it will be like that for a while probably.
The main purpose for now is to help development, but it could be refined and promoted to production eventually.

For now it's not a real command, in particular it doesn't do any compute, so it cannot be chained with further commands.
We could make it a real command, with evaluators and all the rest, but now the planning logic is very coupled into EsqlSession, so if we want EXPLAIN to be really representative of what happens in there, we'll have to keep it in there. Or refactor how planning works.

The syntax is EXPLAIN ( <query> ) and produces information about parsed query, optimized logical plan and optimized physical plan

eg.

explain ( row a = 1 )

{
    "took": 281,
    "is_partial": false,
    "documents_found": 0,
    "values_loaded": 0,
    "columns": [
        {
            "name": "role",
            "type": "keyword"
        },
        {
            "name": "type",
            "type": "keyword"
        },
        {
            "name": "plan",
            "type": "keyword"
        }
    ],
    "values": [
        [
            "coordinator",
            "parsedPlan",
            "Row[[1[INTEGER] AS a#2]]"
        ],
        [
            "coordinator",
            "optimizedLogicalPlan",
            "Limit[1000[INTEGER],false]\n\\_LocalRelation[[a{r}#2],[IntVectorBlock[vector=ConstantIntVector[positions=1, value=1]]]]"
        ],
        [
            "coordinator",
            "optimizedPhysicalPlan",
            "LimitExec[1000[INTEGER],4]\n\\_LocalSourceExec[[a{r}#2],[IntVectorBlock[vector=ConstantIntVector[positions=1, value=1]]]]"
        ]
    ]
}

I'm using round parentheses (and not square brackets, as in the original syntax) just because FORK is already using them for subqueries, but it's something we can change.

[luigidellaquila]

Moving it to dev mode. It didn't work before, so there is really no reason to have it in the GA grammar

[luigidellaquila]

Aligned with FORK

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[idegtiarenko]

👍 for making int snapshot only until that is properly finalized

[ivancea]

LGTM. And surely, dev mode suits it better than GA

[ivancea]

I see this RP is missing from some modes:

• SHOW
• JOIN

Now, JOIN technically would fail anyway, as the "ON" token changes to expression mode, so the question is if we want it to fail at lexer or parser level (?).
My guess is that a query like EXPLAIN (FROM index | LOOKUP JOIN lookup) would fail in different ways depending on this, but I'm not sure of which error we usually prefer

[luigidellaquila]

My actual concern is that every time we add a new command we'll have to do all this push/pop dance, but that's a broader topic...
About the error message, I'm not sure either

[ivancea]

What happens if we add an EXPLAIN inside an EXPLAIN? I fear it could fail in the logical-to-physical conversion with a weird error. I would add a test of that here too

[luigidellaquila]

Today it's forbidden on purpose

[ivancea]

Should we pass these vars (explainMode, and the plan strings) as parameters instead? If I'm not wrong, calling session.execute() twice will leak those vars and break the second query. We're not doing that I think, but probably worth avoiding it (?)

[ivancea]

Just saw that the Explain class has some old commented code. I would also remove it and clean it up a bit, as it looks ancient.
Also, it has an output(), which looks wrong now. It's probably better to throw an exception there as it's unused now