Skip to content

Update some sentry_app queries to use the read replica #93667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 17, 2025

Conversation

kneeyo1
Copy link
Contributor

@kneeyo1 kneeyo1 commented Jun 16, 2025

This updates some of the queries to point to the control db replica.
see #93081, breaking that change apart so we can better see the effects of these on the db.

@kneeyo1 kneeyo1 requested review from a team as code owners June 16, 2025 22:24
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 16, 2025
]

def get_sentry_app_by_id(self, *, id: int) -> RpcSentryApp | None:
try:
sentry_app = SentryApp.objects.get(id=id)
sentry_app = SentryApp.objects.using_replica().get(id=id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So on the surface, this makes sense and doesn't seem to have an effect on the calling code, but I do have a couple of questions:

  • Do we have an idea of how much replication lag we'll have to deal with on average?
  • Is there maybe some way in the RPC name itself we can denote that this is using a replica? Having an RPC exhibit eventually consistent behavior can be pretty jarring from the developer perspective if you're not expecting it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going through datadog the average replication delay is around 100 MS for this db.

I could change the function signature to more explicitly call out that this is reading from the db replica, if you'd like?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going through datadog the average replication delay is around 100 MS for this db.

Got it, that's not too bad then.

Re: the RPC name, I don't want to create a ton more work for you since this would require multiple PRs to accomplish, but I do wonder if philosophically we need be more explicit about when we are pushing services calls to read from replicas. Just something to consider.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. for now I can add some docs around the function signature to indidcate that this function is reading from a replica?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A docstring would be a good start. I wouldn't recommend renaming methods as it is a complicated process.

This RPC is mostly called from regions in Endpoint.convert_args(), there is some risk of read-your-writes but the endpoints where it is used are going to be used infrequently.

Copy link
Member

@markstory markstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given where this RPC method is used, I think the risk is low, but also the gains will be low as the throughput is low.

]

def get_sentry_app_by_id(self, *, id: int) -> RpcSentryApp | None:
try:
sentry_app = SentryApp.objects.get(id=id)
sentry_app = SentryApp.objects.using_replica().get(id=id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A docstring would be a good start. I wouldn't recommend renaming methods as it is a complicated process.

This RPC is mostly called from regions in Endpoint.convert_args(), there is some risk of read-your-writes but the endpoints where it is used are going to be used infrequently.

@kneeyo1 kneeyo1 merged commit 4fa42e2 into master Jun 17, 2025
66 checks passed
@kneeyo1 kneeyo1 deleted the kneeyo1/control-changes-pt1 branch June 17, 2025 16:30
billyvg pushed a commit that referenced this pull request Jun 18, 2025
This updates some of the queries to point to the control db replica. 
see #93081, breaking that change apart so we can better see the effects
of these on the db.
andrewshie-sentry pushed a commit that referenced this pull request Jun 19, 2025
This updates some of the queries to point to the control db replica. 
see #93081, breaking that change apart so we can better see the effects
of these on the db.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants