Skip to content

[8.18] [8.x] ESQL: supplement docs on LIMIT (#125967) #126395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/reference/esql/esql-limitations.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
[[esql-max-rows]]
=== Result set size limit

By default, an {esql} query returns up to 1000 rows. You can increase the number
By default, an {esql} query returns up to 1,000 rows. You can increase the number
of rows up to 10,000 using the <<esql-limit>> command.
include::processing-commands/limit.asciidoc[tag=limitation]

Expand Down
50 changes: 40 additions & 10 deletions docs/reference/esql/processing-commands/limit.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,54 @@ The maximum number of rows to return.
The `LIMIT` processing command enables you to limit the number of rows that are
returned.
// tag::limitation[]
Queries do not return more than 10,000 rows, regardless of the `LIMIT` command's
value.
For instance,
```
FROM index | WHERE field = "value"
```
is equivalent to:
```
FROM index | WHERE field = "value" | LIMIT 1000
```

This limit only applies to the number of rows that are retrieved by the query.
Queries and aggregations run on the full data set.
Queries do not return more than 10,000 rows, regardless of the `LIMIT` command’s value. This is a configurable upper limit.

To overcome this limitation:

* Reduce the result set size by modifying the query to only return relevant
data. Use <<esql-where>> to select a smaller subset of the data.
* Shift any post-query processing to the query itself. You can use the {esql}
<<esql-stats-by>> command to aggregate data in the query.
* Reduce the result set size by modifying the query to only return relevant data. Use <<esql-where>> to select a smaller subset of the data.
* Shift any post-query processing to the query itself. You can use the {esql} <<esql-stats-by>> command to aggregate data in the query.

The default and maximum limits can be changed using these dynamic cluster
settings:
The upper limit only applies to the number of rows that are output by the query, not to the number of documents it processes: the query runs on the full data set.

Consider the following two queries:
```
FROM index | WHERE field0 == "value" | LIMIT 20000
```
and
```
FROM index | STATS AVG(field1) BY field2 | LIMIT 20000
```

In both cases, the filtering by `field0` in the first query or the grouping by `field2` in the second is applied over all the documents present in the `index`, irrespective of their number or indexes size. However, both queries will return at most 10,000 rows, even if there were more rows available to return.

The default and maximum limits can be changed using these dynamic cluster settings:

* `esql.query.result_truncation_default_size`
* `esql.query.result_truncation_max_size`

However, doing so involves trade-offs. A larger result-set involves a higher memory pressure and increased processing times; the internode traffic within and across clusters can also increase.

These limitations are similar to those enforced by the <<paginate-search-results>>.

[%header.monospaced.styled,format=dsv,separator=|]
|===

Functionality | Search | {esql}
Results returned by default | 10 | 1.000
Default upper limit | 10,000 | 10,000
Specify number of results | `size` | `LIMIT`
Change default number of results | n/a | esql.query.result_truncation_default_size
Change default upper limit | index-max-result-window | esql.query.result_truncation_max_size
|===
// end::limitation[]

*Example*
Expand Down