ESQL: Document LU JOIN/MV_EXPAND not respecting SORT #127718
Conversation
alex-spies
added
>docs
elasticsearchmachine
added
Team:Docs
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
| @@ -45,6 +45,12 @@ results, the output will contain one row for each matching combination. | |||
| In case of name collisions, the newly created columns will override existing columns. | |||
| :::: | |||
|
|
|||
| ::::{warning} | |||
| The output rows produced by `LOOKUP JOIN` can be in any order and may not | |||
There was a problem hiding this comment.
@leemthompo , should we also place this warning in the more in-depth landing page for lookup join?
There was a problem hiding this comment.
I think we are running the risk of those two pages becoming undifferentiated and therefore hard to maintain, if we repeat everything in both. Also adding admonitions like this doesn't scale.
We could consider moving this warning and the tip above it into the landing page under a new heading like Usage notes. And then we can link to that new section of the landing page from here with a message like "Refer to Usage notes for important information about using the LOOKUP JOIN command"— instead of repeating admonitions on both pages. This will be cleaner, more future-proof, and easier to maintain.
WDYT?
Relates #121884.
We currently cannot run all queries that have a
SORTbefore eitherLOOKUP JOINorMV_EXPAND. This is because we cannot push down these commands pastSORTs without breaking the sort order in some cases. We may or may not want to choose to push down in these cases in the future, even though this affects sort order.Moreover, further work and generalizations of
LOOKUP JOINmay be even less compatible with sorting - e.g. if we generalize lookup indices to have multiple shards residing on different nodes.To keep all possibilities open, let's just document that we give no guarantees whatsoever; we can revisit this later when needed.