-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Text field block loader properly handles null values from delegate #127525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Hi @lkts, I've created a changelog YAML for you. |
Pinging @elastic/es-analytical-engine (Team:Analytics) |
Closes #126427. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's worth updating the description to say that this will return the null_value
from the text fields and probably putting a WARNING
about this somewhere in the docs for synthetic source or text fields. It's weird, but it's the least weird choice.
Pinging @elastic/es-docs (Team:Docs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks Sasha!
I'll wait for a review from docs team. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some small suggestions for you from a docs pov
::::{note} | ||
Synthetic source of the `text` field will have the same [modifications](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) as a `keyword` field in this case. | ||
|
||
These modifications can impact usage of `text` fields: | ||
* Reordering text fields can have an effect on [phrase](/reference/query-languages/query-dsl/query-dsl-match-query-phrase.md) and [span](/reference/query-languages/query-dsl/span-queries.md) queries. See the discussion about [`position_increment_gap`](/reference/elasticsearch/mapping-reference/position-increment-gap.md) for more detail. You can avoid this by making sure the `slop` parameter on the phrase queries is lower than the `position_increment_gap`. This is the default. | ||
* Handling of `null` values is different. `text` fields ignore `null` values but `keyword` fields support replacing `null`s with a value specified in the `null_value` parameter. This replacement will be represented in synthetic source. | ||
:::: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you've reframed the entire section to be about keyword
, so I'd take this out of a note
::::{note} | |
Synthetic source of the `text` field will have the same [modifications](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) as a `keyword` field in this case. | |
These modifications can impact usage of `text` fields: | |
* Reordering text fields can have an effect on [phrase](/reference/query-languages/query-dsl/query-dsl-match-query-phrase.md) and [span](/reference/query-languages/query-dsl/span-queries.md) queries. See the discussion about [`position_increment_gap`](/reference/elasticsearch/mapping-reference/position-increment-gap.md) for more detail. You can avoid this by making sure the `slop` parameter on the phrase queries is lower than the `position_increment_gap`. This is the default. | |
* Handling of `null` values is different. `text` fields ignore `null` values but `keyword` fields support replacing `null`s with a value specified in the `null_value` parameter. This replacement will be represented in synthetic source. | |
:::: | |
In this case, the synthetic source of the `text` field will have the same [modifications](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) as a `keyword` field. | |
These modifications can impact usage of `text` fields: | |
* Reordering text fields can have an effect on [phrase](/reference/query-languages/query-dsl/query-dsl-match-query-phrase.md) and [span](/reference/query-languages/query-dsl/span-queries.md) queries. See the discussion about [`position_increment_gap`](/reference/elasticsearch/mapping-reference/position-increment-gap.md) for more details. You can avoid this by making sure the `slop` parameter on the phrase queries is lower than the `position_increment_gap`. This is the default. | |
* Handling of `null` values is different. `text` fields ignore `null` values, but `keyword` fields support replacing `null` values with a value specified in the `null_value` parameter. This replacement is represented in synthetic source. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imo, it would be good to keep an example here because it's the actual synthetic part. not critical, but the store
example is less important I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't want to repeat the same example as keyword
has. Do you think it's helpful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's intuitive for people to look for the "core" example on the keyword page ... they'd likely just scroll down to the synthetics section and copy whatever is there/assume whatever example is there represents the happy path. you could consider using a snippet to avoid maintaining two examples.
I'm also ok with this merging as is if you're not super worried about it 🤷
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that we don't actually have an example with a multi-field i'll go ahead and add it. Thanks for the feedback.
Co-authored-by: shainaraskas <[email protected]>
Co-authored-by: shainaraskas <[email protected]>
Co-authored-by: shainaraskas <[email protected]>
💔 Backport failed
You can use sqren/backport to manually backport by running |
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
See analysis in #126427 (comment).
This change makes
text
fields that have akeyword
multi-field withnull_value
correctly returnnull_value
instead ofnull
when there are onlynull
values in the document. This is already true fornull
s mixed with regular value (not considering whether that is intentional or not).F.e. indexing
["hey", null]
with a multi-field that hasnull_value: "NULL"
results intext
block loader returning["hey", "NULL"]
. This PR makes[null, null]
returns["NULL", "NULL"]
instead of nothing.