Inline aggregation logic into QueryPhaseResultConsumer #126730
+20
−29
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This refactoring removes some needless indirection but more importantly, it sets up a significant improvement to aggs: passing them to the reducer as they become available and without retaining them in a list (serialized or materialized) any longer than necessary. I chose to do this step first to make that change shorter and more focussed as it will involve a bit of non-trivial concurrency.
elasticsearchmachine
added
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
| } finally { | ||
| toConsume.forEachRemaining(QuerySearchResult::releaseAggs); | ||
| } | ||
| } | ||
|
|
||
| private static AggregatorsReducer createReducer(int resultSetSize, AggregationReduceContext reduceContext, InternalAggregations first) { |
There was a problem hiding this comment.
The follow-up from this would be to create the reducer up-front and in the size of either the final resultset (if no partial reduces come into play) or the size of the next partial reduce and consume aggregations directly into the aggregator as results come in.
I chose to do this in 2 steps as making this performance while receiving results concurrently will require forcing all results onto a single thread which is little code but non-trivial to reason about :)
This change gets us close enough to make it little code though :)
elasticsearchmachine
added
the
Team:Search Foundations
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
…umer' into inline-into-query-phase-res-consumer
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refactoring removes some needless indirection but more importantly, it sets up a significant improvement to aggs: passing them to the reducer as they become available and without retaining them in a list (serialized or materialized) any longer than necessary.
I chose to do this step first to make that change shorter and more focussed as it will involve a bit of non-trivial concurrency.