ESQL - transport version change to support TSDB metadata #129649
Conversation
not-napoleon
added
>non-issue
:StorageEngine/TSDB
…-to-es-field' into add-tsdb-type-to-es-field
|
🔍 Preview links for changed docs: 🔔 The preview site may take up to 3 minutes to finish building. These links will become live once it completes. |
Conflicts: docs/reference/query-languages/esql/kibana/definition/functions/knn.json docs/reference/query-languages/esql/kibana/docs/functions/knn.md server/src/main/java/org/elasticsearch/TransportVersions.java
| @@ -214,10 +216,11 @@ private static EsField fieldWithRecursiveChildren(int depth, int childrenPerLeve | |||
| * A single root with 9 children, each of which has 9 children etc. 6 levels deep. | |||
| */ | |||
| public void testDeeplyNestedFields() throws IOException { | |||
| ByteSizeValue expectedSize = ByteSizeValue.ofBytes(9425494); | |||
| ByteSizeValue expectedSize = ByteSizeValue.ofBytes(10023365); | |||
There was a problem hiding this comment.
There will be a small overhead for use cases with many fields, but I think this should be fine.
elasticsearchmachine
added
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
I think it's going to depend on the specific conflict. If a field is mapped as a dimension in one index and a metric in another, that seems like it has to be an error. I don't see a sensible way to proceed with that query. Going from no metadata type to dimension should just promote to dimension (that's the "label" use case). No metadata to metric I'm not sure about. I can't think of a case where that's not an error, but I'm not sure there isn't one. At any rate, we should probably have more discussion about this as we are building this feature. The conflict case is non-trivial. |
I've seen metrics that are not marked as such, but they're simple |
#129649 introduced a duplicate transport version. This fixes it.
) Relates to elastic#128621 This is a first step in making the ES|QL query planner aware of TSDB Dimensions and Metric field metadata. This is purposefully small to only touch the serialization change we'll need for this. The plan is get the TSDB metadata field type out of Field Caps and to store this information on EsField. This PR adds a place to store such a field, and adds it to the serialization for EsField and its sub-classes. As of this PR, we don't do anything with this data. That's intentional, to minimize the footprint of the transport version change. Further PRs in this project will load and act on this data. I've added some constructors here to minimize the number of files I'm touching in this PR. I hope that as we begin loading this data (as opposed to just defaulting it right now) we can get rid of some of these default value constructors.
elastic#129649 introduced a duplicate transport version. This fixes it.
Relates to #128621
This is a first step in making the ES|QL query planner aware of TSDB Dimensions and Metric field metadata. This is purposefully small to only touch the serialization change we'll need for this. The plan is get the TSDB metadata field type out of Field Caps and to store this information on
EsField. This PR adds a place to store such a field, and adds it to the serialization forEsFieldand its sub-classes.As of this PR, we don't do anything with this data. That's intentional, to minimize the footprint of the transport version change. Further PRs in this project will load and act on this data. I've added some constructors here to minimize the number of files I'm touching in this PR. I hope that as we begin loading this data (as opposed to just defaulting it right now) we can get rid of some of these default value constructors.