Skip to content

Add optional error-on-integer-overflow setting for QuerySet filters #55

Open
@bubusuke

Description

@bubusuke

(English is not my native language.)

Code of Conduct

  • I agree to follow Django's Code of Conduct

Feature Description

Adds an opt-in Django setting that raises a new EmptyResultSetDueToIntegerOverflow exception whenever arithmetic in a QuerySet filter would overflow an integer column, instead of silently returning an empty result.
This surfaces hidden overflows and simplifies debugging and migrations from Django 4.x to 5.x.

Problem

  • Problem

    • In Django 5.0 the behaviour for arithmetic that overflows an integer column changed: a filter that would overflow now always returns an empty QuerySet instead of being transparently cast to a wider integer type (the historical behaviour on most back-ends).
    • It makes migrations from Django 4.x harder to verify—tests may “pass” yet silently return fewer (or zero) rows. It is also difficult for developers on 5.x to notice when an overflow is happening because the condition is swallowed inside the ORM.
  • Benefits

    • Safe migrations – Projects upgrading from 4.x can enable the flag in CI to surface every place where arithmetic now truncates results.

    • Debuggability – Developers on 5.x who run into an unexpected empty queryset get an immediate stack-trace pointing at the offending expression.

    • Backwards-compatible – Opt-in toggle preserves the 5.0 behaviour by default; no existing code breaks unless the project explicitly asks for stricter checking.

Request or proposal

proposal

Additional Details

Add a new setting ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET to switch behavior to raise EmptyResultSet or raising EmptyResultSetDueToIntegerOverflow exception if the QuerySet filter's operation is about to overflow an integer column. (See detailed flow in the implementation suggestions).

Implementation Suggestions

  • Add new parameter.
# settings.py
ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET = True  # default: False
  • Add new exception
class EmptyResultSetDueToIntegerOverflow(Exception):
    """Raised when an integer overflow is detected while building a WHERE
    clause.  Subclasses plain Exception so it is *not* caught by existing
    EmptyResultSet handlers."""
  • Raise it at the source

In django.db.models.lookups.IntegerFieldOverflow (or equivalent part of the lookup machinery), raise EmptyResultSetDueToIntegerOverflow instead of EmptyResultSet.

  • Handle it centrally

In django.db.models.sql.where.WhereNode.as_sql() wrap the call to compiler.compile(child)

try:
    ...
except EmptyResultSetDueToIntegerOverflow:
    if settings.ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET:
        raise
    empty_needed -= 1  # current silent-empty logic

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Idea

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions