Skip to content

synthetic_source_keep: "arrays" overtriggers for fields that parse arrays #126155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lkts opened this issue Apr 2, 2025 · 3 comments
Open

synthetic_source_keep: "arrays" overtriggers for fields that parse arrays #126155

lkts opened this issue Apr 2, 2025 · 3 comments
Assignees
Labels

Comments

@lkts
Copy link
Contributor

lkts commented Apr 2, 2025

This can be reproduced by this test https://github.com/elastic/elasticsearch/blob/main/server/src/test/java/org/elasticsearch/index/mapper/blockloader/GeoPointFieldBlockLoaderTests.java#L77. geo_point fields have custom handling of arrays and therefore synthetic_source_keep: "arrays" should not apply to it (arrays have special meaning so we don't know how to exactly preserve them). However we put values of geo_point field in _ignored_source in some cases.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@lkts
Copy link
Contributor Author

lkts commented Apr 2, 2025

The logic is here https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/index/mapper/DocumentParser.java#L463. I think we assume that this code path only handles singular values but we actually end up here when parsing fields with custom array handling because of https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/index/mapper/DocumentParser.java#L627.

@jdcryans
Copy link

jdcryans commented May 1, 2025

@parkertimmins welcome to the team, here's something to look at!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants