Skip to content

[DOCS][8.x] Update ESQL metadata fields page #129996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 26, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 72 additions & 23 deletions docs/reference/esql/metadata-fields.asciidoc
Original file line number Diff line number Diff line change
@@ -1,41 +1,71 @@
[[esql-metadata-fields]]
=== {esql} metadata fields

++++
<titleabbrev>Metadata fields</titleabbrev>
++++

{esql} can access <<mapping-fields, metadata fields>>. The currently
supported ones are:
{esql} can access <<mapping-fields, metadata fields>>.

* <<mapping-index-field,`_index`>>: the index to which the document belongs.
The field is of the type <<keyword, keyword>>.
To access these fields, use the `METADATA` directive with the <<esql-from,`FROM`>> source command. For example:

* <<mapping-id-field,`_id`>>: the source document's ID. The field is of the
type <<keyword, keyword>>.
[source,esql]
----
FROM index METADATA _index, _id
----

* `_version`: the source document's version. The field is of the type
<<number,long>>.
[[esql-metadata-fields-available]]
==== Available metadata fields

* <<mapping-ignored-field,`_ignored`>>: the ignored source document fields. The field is of the type
<<keyword,keyword>>.
The following metadata fields are available in {esql}:

* `_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using <<esql-search-functions,full text search functions>>.
[cols="1,1,3"]
|===
|Metadata field |Type |Description

To enable the access to these fields, the <<esql-from,`FROM`>> source command needs
to be provided with a dedicated directive:
|<<mapping-id-field,`_id`>>
|<<keyword, keyword>>
|Unique document ID.

[source,esql]
----
FROM index METADATA _index, _id
----
|<<mapping-ignored-field,`_ignored`>>
|<<keyword, keyword>>
|Names every field in a document that was ignored when the document was indexed.

|<<mapping-index-field,`_index`>>
|<<keyword, keyword>>
|Index name.

|`_index_mode`
|<<keyword, keyword>>
|<<index-mode-setting,Index mode>>. For example: `standard`, `lookup`, or `logsdb`.

|`_score`
|<<number,`float`>>
|Query relevance score (when enabled). Scores are updated when using <<esql-search-functions,full text search functions>>.

Metadata fields are only available if the source of the data is an index.
Consequently, `FROM` is the only source commands that supports the `METADATA`
directive.
|<<mapping-source-field,`_source`>>
|Special `_source` type
|Original JSON document body passed at index time (or a reconstructed version if <<synthetic-source,synthetic `_source`>> is enabled).

Once enabled, these fields will be available to subsequent processing commands, just
like other index fields:
|`_version`
|<<number,`long`>>
|Document version number
|===

[[esql-metadata-fields-usage]]
==== Usage and limitations

- Metadata fields are only available when the data source is an index
- The `_source` type is not supported by functions
- Only the `FROM` command supports the `METADATA` directive
- Once enabled, metadata fields work like regular index fields

[[esql-metadata-fields-examples]]
==== Examples

[[esql-metadata-fields-examples-basic]]
===== Basic metadata usage

Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:

[source.merge.styled,esql]
----
Expand All @@ -46,6 +76,9 @@ include::{esql-specs}/metadata.csv-spec[tag=multipleIndices]
include::{esql-specs}/metadata.csv-spec[tag=multipleIndices-result]
|===

[[esql-metadata-fields-examples-aggregations]]
===== Metadata fields and aggregations

Similar to index fields, once an aggregation is performed, a
metadata field will no longer be accessible to subsequent commands, unless
used as a grouping field:
Expand All @@ -58,3 +91,19 @@ include::{esql-specs}/metadata.csv-spec[tag=metaIndexInAggs]
|===
include::{esql-specs}/metadata.csv-spec[tag=metaIndexInAggs-result]
|===

[[esql-metadata-fields-examples-score]]
===== Sort results by search score

[source,esql]
----
FROM products METADATA _score
| WHERE MATCH(description, "wireless headphones")
| SORT _score DESC
| KEEP name, description, _score
----

[TIP]
====
Refer to <<esql-for-search>> for more information on relevance scoring and how to use `_score` in your queries.
====