Skip to content

Conversation

cjonesy
Copy link
Contributor

@cjonesy cjonesy commented Oct 12, 2025

The grid structure endpoint was scanning hundreds or thousands of TaskInstance rows when only a few unique dag_version_id values were needed. With my local tests using mapped tasks, this caused 30+ second load times.

Adding .distinct() reduces the subquery results from potentially 1000s of rows to just 1-5 unique dag_version_ids, dramatically improving performance for DAGs with mapped tasks.

The results from my test using Postgres locally with and without distinct() are below:

WITH distinct():
  Run 1: 37.71ms (3 unique versions)
  Run 2: 34.62ms (3 unique versions)
  Run 3: 35.82ms (3 unique versions)

WITHOUT distinct():
  Run 1: 208.54ms (20000 rows)
  Run 2: 159.77ms (20000 rows)
  Run 3: 189.30ms (20000 rows)

closes #56554


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

The grid structure endpoint was scanning hundreds or thousands of
TaskInstance rows when only a few unique dag_version_id values were
needed. With mapped tasks, this caused 30+ second load times.

Adding .distinct() reduces the subquery results from potentially
1000s of rows to just 1-5 unique dag_version_ids, dramatically
improving performance for DAGs with mapped tasks.
Copy link
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Let's wait for CI turn green

@cjonesy cjonesy mentioned this pull request Oct 12, 2025
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grid View UI slow

2 participants