Skip to content

ES|QL change_point processing command #120998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Feb 10, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
6ea45d9
Grammar for ES|QL change point (with dummy impl)
jan-elastic Jan 24, 2025
eadffb9
pipeline breaker
jan-elastic Jan 24, 2025
f1eb1b2
ChangePointOperator
jan-elastic Jan 27, 2025
7333fcf
Add sorting
jan-elastic Jan 27, 2025
10032c2
basic csv test
jan-elastic Jan 27, 2025
6a62802
conflict
jan-elastic Jan 28, 2025
49600b6
Update docs/changelog/120998.yaml
jan-elastic Jan 28, 2025
77572e9
[CI] Auto commit changes from spotless
elasticsearchmachine Jan 28, 2025
0886289
polish
jan-elastic Jan 28, 2025
97f16e3
Non-long data type
jan-elastic Jan 28, 2025
c2c5911
Move OrderBy/Limit to the logical plan
jan-elastic Jan 29, 2025
cddcfab
fix mem.leak
jan-elastic Jan 30, 2025
14072dc
csv test for reusing column names
jan-elastic Jan 30, 2025
c5fd3cb
Warning indeterminable
jan-elastic Jan 30, 2025
e905a79
capability
jan-elastic Jan 30, 2025
995baad
handle null values
jan-elastic Jan 30, 2025
cd223c3
too much data
jan-elastic Jan 30, 2025
8d0f691
type text->keyword
jan-elastic Jan 31, 2025
ea24893
default timestamp and output columns
jan-elastic Jan 31, 2025
37c3be2
spotless
jan-elastic Jan 31, 2025
1088a09
ChangePointOperatorTests + fix memory leaks
jan-elastic Jan 31, 2025
7f01d41
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Jan 31, 2025
eabbaa1
[CI] Auto commit changes from spotless
elasticsearchmachine Jan 31, 2025
7d30d44
improve test
jan-elastic Jan 31, 2025
ac48a09
add comments/todos
jan-elastic Feb 3, 2025
906ca1a
handle multivalued columns
jan-elastic Feb 3, 2025
33ef895
don't register unserialiazable
jan-elastic Feb 3, 2025
98fa981
surrogate
jan-elastic Feb 3, 2025
535101b
make "too much data" tests readable
jan-elastic Feb 3, 2025
8692df6
more tests
jan-elastic Feb 3, 2025
5cab77b
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 3, 2025
61e24dd
Error handling
jan-elastic Feb 3, 2025
976450e
fix multivalued test
jan-elastic Feb 4, 2025
4885af7
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 4, 2025
047feb7
more name conflict tests
jan-elastic Feb 4, 2025
9278422
[CI] Auto commit changes from spotless
elasticsearchmachine Feb 4, 2025
41057f7
more tests
jan-elastic Feb 4, 2025
f9464de
improve code
jan-elastic Feb 5, 2025
f50c2aa
CSV test for various input key/value types
jan-elastic Feb 5, 2025
8e05959
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 5, 2025
a7c76dd
one more csv test
jan-elastic Feb 5, 2025
bed70ab
Check sortable/numeric for all types
jan-elastic Feb 5, 2025
e81e47e
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 5, 2025
688567b
add null type to testChangePoint_valueNumeric
jan-elastic Feb 6, 2025
22acc9b
more CSV tests
jan-elastic Feb 6, 2025
2b137ca
skip nulls instead of zeroing them
jan-elastic Feb 6, 2025
96e6d6a
error on MV
jan-elastic Feb 6, 2025
f00522a
Test+todo for nicer error message
jan-elastic Feb 6, 2025
12876e0
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 6, 2025
21ec77c
better error msg
jan-elastic Feb 6, 2025
1d3e362
Revert "better error msg"
jan-elastic Feb 6, 2025
dabf260
fix
jan-elastic Feb 6, 2025
459d408
make csv test deterministic
jan-elastic Feb 6, 2025
dbd5838
replace NamedExpression -> Attribute
jan-elastic Feb 10, 2025
4ced5c2
skip MVs + warning
jan-elastic Feb 10, 2025
654f78d
Merge branch 'main' of github.com:elastic/elasticsearch into esql-cha…
jan-elastic Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "better error msg"
This reverts commit 21ec77c.
  • Loading branch information
jan-elastic committed Feb 6, 2025
commit 1d3e3624eb602c7c8b80e7ddaabeacbeddbac9de
Original file line number Diff line number Diff line change
Expand Up @@ -445,9 +445,7 @@ public PlanFactory visitEnrichCommand(EsqlBaseParser.EnrichCommandContext ctx) {
public PlanFactory visitChangePointCommand(EsqlBaseParser.ChangePointCommandContext ctx) {
Source src = source(ctx);
NamedExpression value = visitQualifiedName(ctx.value);
NamedExpression key = ctx.key == null
? new UnresolvedAttribute(src, "@timestamp", "Key column unspecified and default [@timestamp] is missing")
: visitQualifiedName(ctx.key);
NamedExpression key = ctx.key == null ? new UnresolvedAttribute(src, "@timestamp") : visitQualifiedName(ctx.key);
Attribute targetType = new ReferenceAttribute(
src,
ctx.targetType == null ? "type" : visitQualifiedName(ctx.targetType).name(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2021,10 +2021,8 @@ public void testChangePoint() {
var airports = AnalyzerTestUtils.analyzer(loadMapping("mapping-airports.json", "airports"));
assertEquals("1:30: Unknown column [blahblah]", error("FROM airports | CHANGE_POINT blahblah ON scalerank", airports));
assertEquals("1:43: Unknown column [blahblah]", error("FROM airports | CHANGE_POINT scalerank ON blahblah", airports));
assertEquals(
"1:17: Key column unspecified and default [@timestamp] is missing",
error("FROM airports | CHANGE_POINT scalerank", airports)
);
// TODO: nicer error message for missing default column "@timestamp"
assertEquals("1:17: Unknown column [@timestamp]", error("FROM airports | CHANGE_POINT scalerank", airports));
}

public void testChangePoint_keySortable() {
Expand Down