Skip to content

[ES|QL] DATE_DIFF Integer Overflow returns nothing #125457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BenB196 opened this issue Mar 22, 2025 · 1 comment
Open

[ES|QL] DATE_DIFF Integer Overflow returns nothing #125457

BenB196 opened this issue Mar 22, 2025 · 1 comment
Labels
:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) team-discuss

Comments

@BenB196
Copy link

BenB196 commented Mar 22, 2025

Elasticsearch Version

8.17.1

Installed Plugins

No response

Java Version

bundled

OS Version

Container

Problem Description

When you run the DATE_DIFF function that produces a value larger than what can fit into an Integer (DATE_DIFF's response value), nothing is returned.

I'm opening this as a bug for a few reasons, but it's possible that it might also be a feature request:

  1. ES|QL doesn't seem to have any documentation on the standard response one should expect when a value overflow happens; (should it be an error/exception?, should it return null?)
    • For me it took a bit to recognize in a query that I was writing that I was seeing weird results because DATE_DIFF was overflowing
  2. DATE_DIFF being locked to a response of integer rather than long (or possibly definable/dynamic) seems somewhat limiting, especially with the smaller time units and longer date ranges.

Steps to Reproduce

  1. Run an ES|QL query where 2 dates have a diff larger than int in any time unit.

Example:

ROW start_date = "2025-02-14T17:05:36.630Z"::datetime, end_date = "2025-03-22T15:17:56.631Z"::datetime
| EVAL diff = DATE_DIFF("millisecond", start_date, end_date)
  1. Observe that the response does not include anything for diff
{
  "rawResponse": {
    "is_running": false,
    "took": 3,
    "all_columns": [
      {
        "name": "start_date",
        "type": "date"
      },
      {
        "name": "end_date",
        "type": "date"
      },
      {
        "name": "diff",
        "type": "integer"
      }
    ],
    "columns": [
      {
        "name": "start_date",
        "type": "date"
      },
      {
        "name": "end_date",
        "type": "date"
      }
    ],
    "values": [
      [
        "2025-02-14T17:05:36.630Z",
        "2025-03-22T15:17:56.631Z"
      ]
    ]
  }
}
  1. Change the unit to something that doesn't overflow int

Example:

ROW start_date = "2025-02-14T17:05:36.630Z"::datetime, end_date = "2025-03-22T15:17:56.631Z"::datetime
| EVAL diff = DATE_DIFF("second", start_date, end_date)
  1. Observe that a value is included for diff.
{
  "rawResponse": {
    "is_running": false,
    "took": 2,
    "all_columns": [
      {
        "name": "start_date",
        "type": "date"
      },
      {
        "name": "end_date",
        "type": "date"
      },
      {
        "name": "diff",
        "type": "integer"
      }
    ],
    "columns": [
      {
        "name": "start_date",
        "type": "date"
      },
      {
        "name": "end_date",
        "type": "date"
      },
      {
        "name": "diff",
        "type": "integer"
      }
    ],
    "values": [
      [
        "2025-02-14T17:05:36.630Z",
        "2025-03-22T15:17:56.631Z",
        3103940
      ]
    ]
  }
}

Logs (if relevant)

No response

@BenB196 BenB196 added >bug needs:triage Requires assignment of a team area label labels Mar 22, 2025
@iverase iverase added :Analytics/EQL EQL querying and removed needs:triage Requires assignment of a team area label labels Mar 24, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 24, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@iverase iverase added :Analytics/ES|QL AKA ESQL and removed :Analytics/EQL EQL querying labels Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) team-discuss
Projects
None yet
Development

No branches or pull requests

4 participants