ESQL: Preserve single aggregate when all attributes are pruned

[kanoshiou]

In PruneColumns, Aggregate might be optimized to produce a LocalRelation output with an EmptyAttribute. However, the subsequent eval execution does not filter out this EmptyAttribute, which could lead to an additional null column with an empty name appearing in the result.

Closes #126392

[kanoshiou]

AVG appears to function differently compared to other stats functions. Since AVG applies a SubstituteSurrogates rule, it may result in the insertion of a Project, which prevents the issue from being reproducible using the query provided below.

ROW foo = 10
| STATS x = AVG(foo) 
| EVAL x = 13

// before
Limit[1000[INTEGER],false]
\_Eval[[13[INTEGER] AS x]]
  \_Aggregate[[],[AVG(foo{r}#812,true[BOOLEAN]) AS x]]
    \_Row[[10[INTEGER] AS foo]]

// after
Limit[1000[INTEGER],false]
\_Eval[[13[INTEGER] AS x]]
  \_Project[[x{r}#815]]
    \_Eval[[$$SUM$x$0{r$}#818 / $$COUNT$x$1{r$}#819 AS x]]
      \_Aggregate[[],[SUM(foo{r}#812,true[BOOLEAN]) AS $$SUM$x$0, COUNT(foo{r}#812,true[BOOLEAN]) AS $$COUNT$x$1]]
        \_Row[[10[INTEGER] AS foo]]

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[fang-xing-esql]

buildkite test this

[fang-xing-esql]

buildkite test this

[fang-xing-esql]

Thank you for catching this bug @kanoshiou!

I wonder if you have considered the option to limit the scope of change inside the PruneColumns rule, if it is the rule that creates an EmptyAttribute that causes trouble in NamedExpressions.mergeOutputExpressions?

Changing NamedExpressions.mergeOutputExpressions to skip EmptyAttribute fixes the bug, however this method is kind of a low level API that is called by parser, the output() method of LogicalPlans and another rule which might be irrelevant to this bug, it looks to me that the scope of impact of this change might be quite larger than we had expected.

An alternative way that I can think of is limiting the change to the PruneColumns itself, when creating a LocalRelation, avoiding using EmptyAttribute, so that it won't cause NamedExpressions.mergeOutputExpressions to add a null field to the output.

[kanoshiou]

Thank you for your review @fang-xing-esql !

Your concern about NamedExpressions.mergeOutputExpressions is right. Introducing such changes might lead to unexpected bugs in the future due to the wide scope of impact.

I've updated the branch. Could you please review it again?

[fang-xing-esql]

Thank you @kanoshiou! LGTM

[fang-xing-esql]

I was thinking about something like List.of(aggregate.output().get(0)), your way does a similar job too.

[fang-xing-esql]

buildkite test this

[fang-xing-esql]

buildkite test this

[elasticsearchmachine]

💚 Backport successful

Status	Branch	Result
✅	8.x