ESQL: Don't push down filters on the right hand side of an inlinejoin #127383
Conversation
|
Hi @astefan, I've created a changelog YAML for you. |
inlinestats capability)
…ps://github.com/astefan/elasticsearch into skip_filters_pushdown_on_inlinestats_right_side
…skip_filters_pushdown_on_inlinestats_right_side
elasticsearchmachine
added
the
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
…skip_filters_pushdown_on_inlinestats_right_side
|
@elasticmachine run elasticsearch-ci/part-3 |
1 similar comment
|
@elasticmachine run elasticsearch-ci/part-3 |
astefan
added
auto-backport
There was a problem hiding this comment.
This is simple enough, just disable the faulty optimization and be done with it. Thank you @astefan !
I left some thoughts on how we could re-enable this optimization. I believe this hinges on the StubRelation approach, which I think leads to issues that need to be thought about and already had prompted me to open #124752 .
| // TODO: could we do better here about pushing down filters for inlinestats? | ||
| // See also https://github.com/elastic/elasticsearch/issues/127497 | ||
| // Push down past INLINESTATS if the condition is on the groupings |
There was a problem hiding this comment.
I believe we can do better, but the stub relation approach of InlineJoin is getting in the way.
The StubRelation approach also hinges on the subtle idea that the same name ids occuring on the left and right hand side of the join is okay, but I have doubts about that.
Since we're not physically re-using the left hand side output to compute the stats on the right, but we instead run 2 separate (but dependent) queries, I believe it may be simpler to just "copy" the left hand side in place of the stub; this would allow for independent optimization more easily and be more honest, although I haven't thought through all the implications of what I just said.
There was a problem hiding this comment.
Indeed, StubRelation needs a rethinking especially related to Mapper. Thanks @alex-spies
💔 Backport failed
You can use sqren/backport to manually backport by running |
…elastic#127383) * Don't push down filters on the right hand side of an inlinejoin. (cherry picked from commit 4bb4d4b)
Pushing down filters on the left hand side of an inlinestats commands could apparently be beneficial, but the source of the right hand side of the said inlinestats is the same as the left hand side one and this will inherently use the same filters, which is wrong.
For ex,
FROM employees | INLINESTATS max_salary = MAX(salary) by languages | KEEP emp_no, languages, max_salary | WHERE emp_no == 10003would consideremp_no == 10003as eligible to be used likeWHERE emp_no == 10003 | STATS MAX(salary) by languageswhich changes drastically the outcome of thestatscommand.