Skip to content

Emit ordinal output block for values aggregate #127201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 23, 2025

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Apr 22, 2025

Time-series aggregations rely heavily on the values aggregation for collecting grouping values. For example:

TS k8s | STATS max(rate(request)) BY host

is translated to:

TS k8s
| STATS rate(request), VALUES(host) BY _tsid
| STATS max(`rate(request)`) BY host=`VALUES(host)`

We might change how these are executed later, but for now, we need to optimize the values aggregation for BytesRef, especially in cases with low cardinality. This change emits ordinal blocks as the output of the values aggregation, allowing the second aggregation to execute more efficiently. I will also open a PR to handle incoming ordinal blocks for the values aggregation.

@dnhatn dnhatn force-pushed the emit-ordinal-values branch from 9a07845 to 84b187f Compare April 23, 2025 01:30
@dnhatn dnhatn marked this pull request as ready for review April 23, 2025 03:38
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Apr 23, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine
Copy link
Collaborator

Hi @dnhatn, I've created a changelog YAML for you.

@dnhatn
Copy link
Member Author

dnhatn commented Apr 23, 2025

Thanks Kostas!

@dnhatn dnhatn merged commit 874a697 into elastic:main Apr 23, 2025
17 checks passed
@dnhatn dnhatn deleted the emit-ordinal-values branch April 23, 2025 05:30
@dnhatn dnhatn mentioned this pull request Apr 27, 2025
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants