fast_import: allow for deletion of importing timelines #11897
Assignees
Labels
Comments
VladLazar
added
c/storage/pageserver
20 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
May 29, 2025
## Problem Importing timelines can't currently be deleted. This is problematic because: 1. Cplane cannot delete failed imports and we leave the timeline behind. 2. The flow does not support user driven cancellation of the import ## Summary of changes On the pageserver: I've taken the path of least resistance, extended `TimelineOrOffloaded` with a new variant and added handling in the right places. I'm open to thoughts here, but I think it turned out better than I was envisioning. On the storage controller: Again, fairly simple business: when a DELETE timeline request is received, we remove the import from the DB and stop any finalization tasks/futures. In order to stop finalizations, we track them in-memory. For each finalizing import, we associate a gate and a cancellation token. Note that we delete the entry from the database before cancelling any finalizations. This is such that a concurrent request can't progress the import into finalize state and race with the deletion. This concern about deleting an import with on-going finalization is theoretical in the near future. We are only going to delete importing timelines after the storage controller reports the failure to cplane. Alas, the design works for user driven cancellation too. Closes #11897
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Since importing timelines are outside of the normal timeline lifecycle, they currently can't be deleted.
We need deletion for cleaning up of failed imports and user cancellation.
The text was updated successfully, but these errors were encountered: