Incorrect "Sync Already Running" errors

[ttogola]

We use Airbyte Cloud and its api to trigger jobs and react to job completions via webhook notifications. However, the job completion notifications are not a reliable indicator. We always get the following error when we immediately trigger a job for a connection that just completed a previous job:

A synchronization is already in progress for this connection: 409 - {"status":409,"type":"[https://reference.airbyte.com/reference/errors#try-again-later","title":"try-again-later","detail":"Please](https://reference.airbyte.com/reference/errors#try-again-later%22,%22title%22:%22try-again-later%22,%22detail%22:%22Please) try again later.","documentationUrl":null,"data":{"message":"A sync is already running for: 096697e1-f157-41ab-a01d-13d7b2da6420"}}

In other words, airbyte is telling us that a job is complete for a connection but we still can't trigger another job for said connection. We have confirmed that the connection truly has no other job running when this happens. Even the list jobs endpoint confirms this. So there is an inconsistent state on airbyte's side.

This is a big problem, because it prevents us from chaining jobs for a given connection. From our investigation, we typically have to retry up to more than a minute after receiving a job completion webhook, before we can actually trigger a next job.