-
Notifications
You must be signed in to change notification settings - Fork 108
BlockHistoryMigration resilience to consensus out of sync with Pact #2261
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BlockHistoryMigration resilience to consensus out of sync with Pact #2261
Conversation
Created using jj-spr 1.3.6-beta.1
Created using jj-spr 1.3.6-beta.1
Created using jj-spr 1.3.6-beta.1 [skip ci]
Created using jj-spr 1.3.6-beta.1 [skip ci]
rowBlockHash <- runGetS decodeBlockHash h | ||
lookupRanked bhdb rowBlockHeight rowBlockHash >>= \case | ||
Nothing -> do | ||
-- TODO: stop when there's a missing header? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To my understanding this is an invariant of the system and we should fail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah we should probably have a test for this before merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure it's worth checking now. It'd require some big changes to all of the streaming
things we have here to track the current status, and all to detect a form of corruption we've never seen before to just error on it. I would rather have the Pact payload provider check on startup whether there's any corruption in this table.
Sometimes consensus and Pact are out of sync; this is normal and
expected because catchup is not atomic w.r.t. payload providers. In
particular Pact can be ahead of consensus.
However, consensus' store of validated block headers is used to migrate
the BlockHistory table. If the corresponding headers are missing from
consensus, we can't migrate those rows of the BlockHistory table. Thus
we need to avoid erroring out in that case.
Additionally, the rows of all other Pact tables should agree with the
highest BlockHistory row. Thus we need to execute a rewind to that block
at the end of the migration.
Test Plan:
replay on mainnet database.
Change-Id: Id000000074e2d41ff0d0254f3d31e9c712adc6e5