[ES|QL] TO_IP Doesn't properly handle 0 padded IPv4 Addresses

[BenB196]

Elasticsearch Version

8.17.1

Installed Plugins

No response

Java Version

bundled

OS Version

Container

Problem Description

When attempting to convert a string to IP via TO_IP that has 0 padding (ex: 001.001.001.001) does not correctly process the IP. 0 padding is generally considered a valid way to represent IP addresses, and most system (ex: Windows), will convert 001.001.001.001 -> 1.1.1.1.

Steps to Reproduce

1. Create an ES|QL function like:

ROW str = "001.001.001.001"
| EVAL ip = TO_IP(str)

2. Observe the response doesn't contain the ip value:

{
  "rawResponse": {
    "is_running": false,
    "took": 1,
    "all_columns": [
      {
        "name": "str",
        "type": "keyword"
      },
      {
        "name": "ip",
        "type": "ip"
      }
    ],
    "columns": [
      {
        "name": "str",
        "type": "keyword"
      }
    ],
    "values": [
      [
        "001.001.001.001"
      ]
    ]
  }
}

Expect the response to contain the ip value of 1.1.1.1

This problem can kind of be worked around using REPLACE with regexp

ROW str = "010.10.100.000"
| EVAL ip = TO_IP(REPLACE(str, "(?<=^|\\.)0{0,2}", ""))

But it isn't the best user experience, and I'm sure isn't "free" from a query overhead perspective.

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[nik9000]

I had to dig a bit and found that we explicitly ignore leading 0s. The original reasoning came from: https://github.com/google/guava/blob/master/guava/src/com/google/common/net/InetAddresses.java#L356-L357

    // Disallow leading zeroes, because no clear standard exists on
    // whether these should be interpreted as decimal or octal.

ESQL doesn't have to follow the same rules as elasticsearch uses, but that's why we reject these.

[nik9000]

You do get a warning, though it doesn't mention leading 0s:

< Warning: 299 Elasticsearch-9.1.0-81ef7ddd5352b05b53f0de499a46c7878cee1575 "Line 1:41: evaluation of [TO_IP(str)] failed, treating result as null. Only first 20 failures recorded."
< Warning: 299 Elasticsearch-9.1.0-81ef7ddd5352b05b53f0de499a46c7878cee1575 "Line 1:41: java.lang.IllegalArgumentException: '001.001.001.001' is not an IP string literal."

Looking around the internet it seems like most tools interpret it as octal.

[nik9000]

I've asked for a look from some members of Elastic's security solutions team. They'll have a much more educated opinion than mine. Maybe we want an option so you can pick how leading 0s work?

[BenB196]

Thanks @nik9000 yeah, based on your previous comment, I was going to suggest potentially adding an optional arg to the function to let the user decide if they're dealing with decimal or octal.

[nik9000]

Talked to @ninoslavmiskovic and he talked to his folks and told me:

• It’s a rare but real issue.
• No strong objections to supporting normalization, but:
• It must be explicit to avoid incorrect assumptions.
• It should not replace strict parsing by default.