Skip to content

Conversation

@KiudLyrl
Copy link

Add support to get block ordered by their height.

If a block from the blockchain is too early (blockchain gives us block 75 while we are only at height 55 for example), then this block is stored in memory and at each iteration the code will check if the next block, according to height, is not already available in the "early_block_pool' before trying to make a disk access.

if the next block is available in the pool the disk acess won't be done, the block is yielded and removed from the early block pool.

It is now possible to get the block ordered by their height,
to do so the unordered block that come early are stored
untill they are needed then deleted.

from block 0 to 150000 the maximum number of blocks
in memory was around 800.
Block height is now filled by the get_ordered_blocks()
Blockchain method.
@brannondorsey
Copy link
Contributor

It would seem that this method doesn't actually guarantee that each block will be a subsequent block from the last one.

@brannondorsey
Copy link
Contributor

I've fixed this bug and updated the code to use Python3 in a PR to @LoudMurmur's branch: https://github.com/LoudMurmur/python-bitcoin-blockchain-parser/pull/1. @LoudMurmur you can review and merge, or @alecalve if you would like I can make a PR with these fixes to this repo directly.

@alecalve, it would seem that there are three solutions to this get_ordered_blocks() problem.

I've tested #17 and it seems to corrupt the .bitcoin/blocks/index in it's current state (at least on my machine), causing bitcoin-core to re-index the entire blockchain each time it runs (which takes FOREVER). #20 has a bug with block.height which I've fixed in https://github.com/LoudMurmur/python-bitcoin-blockchain-parser/pull/1. I haven't tried #15.

@alecalve
Copy link
Owner

alecalve commented Feb 7, 2018

I had the corruption issue indeed when trying it. I think using a more modern leveldb library would fix it, since it would also handle locking, which this PR doesn't.

@brannondorsey
Copy link
Contributor

I was in email correspondence with the author of that fork and he has a hunch that that is correct.

@alecalve
Copy link
Owner

Fixed by #28

@alecalve alecalve closed this Feb 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants