[Bug]: 'Result window is too large, from + size must be less than or equal to: [10000] but was [12000]

[RodriguezZhang]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

RAGFlow workspace code commit ID

commit 37075ea

RAGFlow image version

v0.18.0-99-g7d462d7c full

Other environment information

arm64

Actual behavior

{"ragflow_4b47c52612c811f099450242c0a8e006":{"settings":{"index":{"routing":{"allocation":{"include":{"_tier_preference":"data_content"}}},"refresh_interval":"1000ms","number_of_shards":"2","provided_name":"ragflow_4b47c52612c811f099450242c0a8e006","similarity":{"scripted_sim":{"type":"scripted","script":{"source":"double idf = Math.log(1+(field.docCount-term.docFreq+0.5)/(term.docFreq + 0.5))/Math.log(1+((field.docCount-0.5)/1.5)); return query.boost * idf * Math.min(doc.freq, 1);"}}},"creation_date":"1748698477769","number_of_replicas":"0","uuid":"le_ttJBeQ6WnYcUeYA0ZFA","version":{"created":"8500003"}}}},"ragflow_d24b8ef2129811f0928b0242c0a89006":{"settings":{"index":{"routing":{"allocation":{"include":{"_tier_preference":"data_content"}}},"refresh_interval":"1000ms","number_of_shards":"2","provided_name":"ragflow_d24b8ef2129811f0928b0242c0a89006","similarity":{"scripted_sim":{"type":"scripted","script":{"source":"double idf = Math.log(1+(field.docCount-term.docFreq+0.5)/(term.docFreq + 0.5))/Math.log(1+((field.docCount-0.5)/1.5)); return query.boost * idf * Math.min(doc.freq, 1);"}}},"max_result_window":"500000","creation_date":"1743911103126","number_of_replicas":"0","uuid":"EK_9f8DgSOqoBT6ccattfw","version":{"created":"8500003"}}}},"ragflow_8888aa52152211f0b2d10242c0a87006":{"settings":{"index":{"routing":{"allocation":{"include":{"_tier_preference":"data_content"}}},"refresh_interval":"1000ms","number_of_shards":"2","provided_name":"ragflow_8888aa52152211f0b2d10242c0a87006","similarity":{"scripted_sim":{"type":"scripted","script":{"source":"double idf = Math.log(1+(field.docCount-term.docFreq+0.5)/(term.docFreq + 0.5))/Math.log(1+((field.docCount-0.5)/1.5)); return query.boost * idf * Math.min(doc.freq, 1);"}}},"max_result_window":"500000","creation_date":"1744600061833","number_of_replicas":"0","uuid":"7LezeTuSTRqWE2Grj2Mbbg","version":{"created":"8500003"}}}}}

http://192.168.1.20:8080/knowledge/dataset/chunk?id=9055ab0c3e2311f0980c0242ac170006&doc_id=c8d763b23e2311f0ba430242ac170006&page=1200&size=10

page=1200&size=10 12000

BadRequestError('search_phase_execution_exception', meta=ApiResponseMeta(status=400, http_version='1.1', headers={'X-elastic-product': 'Elasticsearch', 'content-type': 'application/vnd.elasticsearch+json;compatible-with=8', 'content-length': '1495'}, duration=0.0032165050506591797, node=NodeConfig(scheme='http', host='es01', port=9200, path_prefix='', headers={'user-agent': 'elasticsearch-py/8.12.1 (Python/3.10.12; elastic-transport/8.12.0)'}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=False, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=True, _extras={})), body={'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'Result window is too large, from + size must be less than or equal to: [10000] but was [12000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.'}], 'type': 'search_phase_execution_exception', 'reason': 'all shards failed', 'phase': 'query', 'grouped': True, 'failed_shards': [{'shard': 0, 'index': 'ragflow_4b47c52612c811f099450242c0a8e006', 'node': 'l4S-yv3_QTesDPTUnwijoQ', 'reason': {'type': 'illegal_argument_exception', 'reason': 'Result window is too large, from + size must be less than or equal to: [10000] but was [12000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.'}}], 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Result window is too large, from + size must be less than or equal to: [10000] but was [12000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Result window is too large, from + size must be less than or equal to: [10000] but was [12000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.'}}}, 'status': 400})

Expected behavior

No response

Steps to reproduce

http://192.168.1.20:8080/knowledge/dataset/chunk?id=9055ab0c3e2311f0980c0242ac170006&doc_id=c8d763b23e2311f0ba430242ac170006&page=1201&size=10

Additional information

No response

[dosubot]

Hi @RodriguezZhang! I'm Dosu and I’m helping the ragflow team.

The error "Result window is too large, from + size must be less than or equal to: [10000] but was [12000]" occurs because Elasticsearch limits the number of results you can retrieve in a single query using the from/size pagination parameters. By default, this limit is set to 10,000 to prevent excessive memory usage.

To handle larger result sets, Elasticsearch recommends using the scroll API, which is designed for efficiently retrieving large datasets in batches.

If you need to increase the limit for a specific index, you must change the index.max_result_window setting directly in Elasticsearch. This setting is not configurable within RAGFlow itself and is not documented or surfaced in RAGFlow’s configuration files or documentation. You can update it using an Elasticsearch command like:

PUT /your_index_name/_settings
{
  "index" : {
    "max_result_window" : 500000
  }
}

Replace your_index_name with the actual index name. Be aware that increasing this limit can have memory and performance implications for your Elasticsearch cluster.

There are no existing issues or documentation in the RAGFlow repository that address this error or provide guidance on changing this setting. Any adjustment must be made directly in your Elasticsearch instance.

To reply, just mention my name @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}