Skip to content

Unexpected floating-point precision issue with scaled_float field #116228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kangquanzhou opened this issue Nov 5, 2024 · 2 comments
Closed

Unexpected floating-point precision issue with scaled_float field #116228

kangquanzhou opened this issue Nov 5, 2024 · 2 comments
Assignees
Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@kangquanzhou
Copy link

Description:
I'm experiencing an unexpected floating-point precision issue when performing aggregations on a scaled_float field in Elasticsearch. I have defined a field with scaled_float type and a scaling_factor of 100. Here are the details:

Elasticsearch Version:
7.14.2

Index Mapping:
{
"mappings": {
"properties": {
"my_field": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}

Sample Data:
POST /my_index/_doc/1
{
"my_field": 0.1
}

POST /my_index/_doc/2
{
"my_field": 0.2
}

Query: I executed the following aggregation to sum the values of my_field:
{
"aggs": {
"total": {
"sum": {
"field": "my_field"
}
}
}
}

Expected Result:
I expected the sum to be 0.3.

Actual Result:
The result returned was 0.30000000000000004.

Questions:
Is there a known issue with floating-point precision when using scaled_float fields?
What are the best practices for managing floating-point precision when performing aggregations in Elasticsearch?
Are there any solutions to round the result to a specific precision?

Thank you for your assistance!

@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Nov 5, 2024
@gwbrown gwbrown added :Analytics/Aggregations Aggregations and removed needs:triage Requires assignment of a team area label labels Nov 7, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 7, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@nik9000 nik9000 self-assigned this Apr 25, 2025
@nik9000
Copy link
Member

nik9000 commented Apr 30, 2025

The short version is: this is known and expected.

The long version is: scaled_float is about storage compression, not about precision. This is discussed a little in the docs, but I'm sure it's not clear enough. Folks see scaled_float and thing "this is for money" but it really isn't.

Worse, aggs always use double precision for almost everything. Certainly sum does. So you are doing what amounts to:

[nik9000@howl elasticsearch]$ python
Python 3.13.2 (main, Feb  5 2025, 08:05:21) [GCC 14.2.1 20250128] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> .1 + .2
0.30000000000000004

Now, ESQL does better. It's SUM is typed - so it'll do a long sum for integers. But we don't have a scaled_float sum. And, even if we did, scaled_float doesn't take enough care to keep everything accurately internally for real banking use.

If you have any ideas on other places to stick this discussion in the docs I'd be happy to open PRs for that, but otherwise, that's where things are.

@nik9000 nik9000 closed this as completed Apr 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

4 participants