Skip to content

ES|QL change_point docs and tech preview #126407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Apr 14, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/reference/query-languages/esql/esql-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ An {{esql}} source command produces a table, typically with data from {{es}}. An

{{esql}} supports these processing commands:

* [`CHANGE_POINT`](#esql-change-point)
* [`DISSECT`](#esql-dissect)
* [`DROP`](#esql-drop)
* [`ENRICH`](#esql-enrich)
Expand Down Expand Up @@ -191,6 +192,67 @@ SHOW INFO
| 8.13.0 | 2024-02-23T10:04:18.123117961Z | 04ba8c8db2507501c88f215e475de7b0798cb3b3 |


## `CHANGE_POINT` [esql-change-point]

::::{warning}
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
::::

`CHANGE_POINT` detects spikes, dips, and change points in a metric.

**Syntax**

```esql
CHANGE_POINT value [ON key] [AS type_name, pvalue_name]
```

**Parameters**

`value`
: The column with the metric in which you want to detect a change point.

`key`
: The column with the key to order the values by. If not specified, `@timestamp` is used.

`type_name`
: The name of the output column with the change point type. If not specified, `type` is used.

`pvalue_name`
: The name of the output column with the p-value that indicates how extreme the change point is. If not specified, `pvalue` is used.

**Description**

`CHANGE_POINT` detects spikes, dips, and change points in a metric. The command adds columns to
the table with the change point type and p-value, that indicates how extreme the change point is
(lower values indicate greater changes).

The possible change point types are:
* `dip`: a significant dip occurs at this change point
* `distribution_change`: the overall distribution of the values has changed significantly
* `spike`: a significant spike occurs at this point
* `step_change`: the change indicates a statistically significant step up or down in value distribution
* `trend_change`: there is an overall trend change occurring at this point

::::{note}
There must be at least 22 values for change point detection. Fewer than 1,000 is preferred.
::::

**Examples**

The following example shows whether there is a day with a significant change in the number of log messages:

```
FROM kibana_sample_data_logs
| STATS message_count=COUNT() BY day=BUCKET(@timestamp, 1 DAY)
| CHANGE_POINT message_count ON day
| WHERE type IS NOT NULL
```

| message_count:long | day:datetime | type:keyword | pvalue:double |
|--------------------|--------------------------|--------------|------------------------|
| 329 | 2025-05-10T00:00:00.000Z | spike | 7.284562237089151e-198 |


## `DISSECT` [esql-dissect]

`DISSECT` enables you to [extract structured data out of a string](/reference/query-languages/esql/esql-process-data-with-dissect-grok.md).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -843,7 +843,7 @@ public enum Cap {
/**
* Support change point detection "CHANGE_POINT".
*/
CHANGE_POINT(Build.current().isSnapshot()),
CHANGE_POINT,

/**
* Fix for https://github.com/elastic/elasticsearch/issues/120817
Expand Down
Loading