Skip to content

Introduce allow_partial_results setting in ES|QL #122890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Mar 5, 2025

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Feb 18, 2025

This change introduces a cluster setting esql.query.allow_partial_results that allows enabling or disabling allow_partial_results in ES|QL at the cluster-wide level. Initially, this setting defaults to false, but it will be switched to true soon. The reason for not changing the default in this PR is that it requires adjusting many tests, which would make the PR too large. Instead, we will adjust the tests incrementally and switch the default when the tests are ready. This cluster setting is useful for falling back to the previous behavior (i.e., disabling allow_partial_results) if users upgrade to the new version and haven't updated their queries.

Also, the default setting can be overridden on a per-request basis via a URL parameter (allow_partial_results) (changed from request body to URL parameter to conform to the proposal).

Relates #122802

@dnhatn dnhatn force-pushed the default-partial-results-rest branch 9 times, most recently from 311e8b0 to ffc7f8f Compare February 20, 2025 16:10
@dnhatn dnhatn force-pushed the default-partial-results-rest branch 2 times, most recently from 1d1bb2b to 25837c5 Compare March 4, 2025 19:08
@dnhatn dnhatn changed the title Allow partial result by default in ES|QL Add allow_partial_results setting in ES|QL Mar 4, 2025
@dnhatn dnhatn force-pushed the default-partial-results-rest branch from 25837c5 to e8873ce Compare March 4, 2025 22:13
@dnhatn dnhatn changed the title Add allow_partial_results setting in ES|QL Introduce allow_partial_results setting in ES|QL Mar 4, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @dnhatn, I've created a changelog YAML for you.

@dnhatn dnhatn mentioned this pull request Mar 4, 2025
10 tasks
@dnhatn dnhatn marked this pull request as ready for review March 4, 2025 22:32
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 4, 2025
@dnhatn dnhatn requested review from smalyshev and quux00 March 4, 2025 22:37
prdoyle and others added 11 commits March 4, 2025 18:52
* Remove duplicate exclusive paths

* Normalize paths in tests to support Windows

* Remove withMode
…ntYamlTestSuiteIT test {yaml=search.vectors/41_knn_search_bbq_hnsw/Vector rescoring has same scoring as exact search for kNN section} elastic#124052
…arty_deployment/Test start deployment fails while model download in progress} elastic#120814
Use the same exception class in more places where single lookup fails
due to the existence of multiple projects. This allows the exception to
be handled similarly, e.g. ignored in
ClusterApplierService#callClusterStateListener
@dnhatn dnhatn added the auto-backport Automatically create backport pull requests when merged label Mar 5, 2025
@dnhatn dnhatn merged commit 2063636 into elastic:main Mar 5, 2025
17 checks passed
@dnhatn dnhatn deleted the default-partial-results-rest branch March 5, 2025 21:48
dnhatn added a commit to dnhatn/elasticsearch that referenced this pull request Mar 5, 2025
This change introduces a cluster setting 
`esql.query.allow_partial_results` that allows enabling or disabling
allow_partial_results in ES|QL at the cluster-wide level. Initially,
this setting defaults to false, but it will be switched to true soon. 
The reason for not changing the default in this PR is that it requires
adjusting many tests, which would make the PR too large. Instead, we
will adjust the tests incrementally and switch the default when the
tests are ready. This cluster setting is useful for falling back to the
previous behavior (i.e., disabling allow_partial_results) if users
upgrade to the new version and haven't updated their queries.

Also, the default setting can be overridden on a per-request basis via a 
URL parameter (allow_partial_results) (changed from request body to URL
parameter to conform to the proposal).

Relates elastic#122802
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

elasticsearchmachine pushed a commit that referenced this pull request Mar 5, 2025
This change introduces a cluster setting 
`esql.query.allow_partial_results` that allows enabling or disabling
allow_partial_results in ES|QL at the cluster-wide level. Initially,
this setting defaults to false, but it will be switched to true soon. 
The reason for not changing the default in this PR is that it requires
adjusting many tests, which would make the PR too large. Instead, we
will adjust the tests incrementally and switch the default when the
tests are ready. This cluster setting is useful for falling back to the
previous behavior (i.e., disabling allow_partial_results) if users
upgrade to the new version and haven't updated their queries.

Also, the default setting can be overridden on a per-request basis via a 
URL parameter (allow_partial_results) (changed from request body to URL
parameter to conform to the proposal).

Relates #122802
georgewallace pushed a commit to georgewallace/elasticsearch that referenced this pull request Mar 11, 2025
This change introduces a cluster setting 
`esql.query.allow_partial_results` that allows enabling or disabling
allow_partial_results in ES|QL at the cluster-wide level. Initially,
this setting defaults to false, but it will be switched to true soon. 
The reason for not changing the default in this PR is that it requires
adjusting many tests, which would make the PR too large. Instead, we
will adjust the tests incrementally and switch the default when the
tests are ready. This cluster setting is useful for falling back to the
previous behavior (i.e., disabling allow_partial_results) if users
upgrade to the new version and haven't updated their queries.

Also, the default setting can be overridden on a per-request basis via a 
URL parameter (allow_partial_results) (changed from request body to URL
parameter to conform to the proposal).

Relates elastic#122802
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants