-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
Background
The current storage of section-bloom data is strongly coupled with the JSON-RPC interface toggle (node.jsonrpc.httpFullNodeEnable), leading to the following issues:
- Irreversible Data Loss: If a user disables the toggle and later re-enables it,
section-bloomdata generated during the disabled period cannot be recovered, causing permanent failures ineth_getLogsqueries. - Confusing Configuration Semantics: A single toggle controls both the data layer and interface layer, blurring behavioral boundaries for users.
- Contradictory Default Behavior: Full nodes should natively support complete log queries, but the current configuration defaults to disabling this functionality.
Data Status (as of 2025-05-29, Mainnet Block Height 72.6 million):
- Theoretical Storage Size: 17.3 GB
- Actual Compressed Size: 12.9 GB (25.4% compression ratio)
Annual growth: ~10 million new blocks, max theoretical increase: 2.4 GB;
Factoring in 25.4% compression, actual storage growth: ~1.79 GB/year.
Rationale
Why should this feature exist?
- Data Integrity: Index data should be decoupled from interface availability to prevent configuration changes from causing data gaps.
- Operational Flexibility: Users may need to temporarily disable interfaces (e.g., during security audits) without compromising data completeness.
What are the use cases?
- Full Node Operation: Preserve
section-bloomdata for future analysis even when the JSON-RPC interface is disabled. - Light Node Optimization: Resource-constrained nodes can disable data writes (
storage.writeSectionBloom=false) to save storage. - Chain Service Providers: DApps require guaranteed completeness of
eth_getLogsresults, unaffected by temporary configuration changes.
Specification
Candidate Solutions Comparison
| Solution | Description | Pros | Cons |
|---|---|---|---|
| Solution 1: Unconditional Writes Remove all write condition checks | Write section-bloom data unconditionally |
Simple implementation (minimal code changes) 100% data integrity | No write disable option (no choice for storage-sensitive users) Violates config controllability |
Solution 2: New Independent Config storage.writeSectionBloom (data toggle) |
Fully decouple data and interface control | User freedom to enable/disable writes Unified control for eth_getFilterChanges and eth_getLogs |
New config affects eth_getFilterChanges Missing historical data requires full sync recovery |
Solution 3: Hybrid Config storage.writeSectionBloom + legacy control |
Decouple persistent data from interface control | Granular write control Compatible with legacy interface logic for eth_getFilterChanges |
Code redundancy Overly complex config granularity |
Test Specification
| Test Scenario | Solution 1 | Solution 2 | Solution 3 |
|---|---|---|---|
| Default configuration | Continuous writes, interface OK | Same | Same |
| Disable writes (if applicable) | N/A | section-bloom writes halted eth_getLogs returns existing data eth_getFilterChanges returns no data |
section-bloom writes halted eth_getLogs returns existing data eth_getFilterChanges returns data |
Scope Of Impact
- Affected Modules:
- Block processing pipeline (
BlockProcessor) - LevelDB storage (
SectionBloomStore) - JSON-RPC interface layer (
EthApi)
- Block processing pipeline (
- Affected Interfaces:
eth_getLogseth_getFilterLogseth_getFilterChanges
Implementation
Approach
Recommended: Solution 2 (New independent config), with steps:
-
Code Decoupling
-
Separate data write logic from interface control
-
if (CommonParameter.getInstance().writeSectionBloom()) { Bloom blockBloom = chainBaseManager.getSectionBloomStore() .initBlockSection(transactionRetCapsule); chainBaseManager.getSectionBloomStore().write(block.getNum()); block.setBloom(blockBloom); }
-
-
Documentation Updates
- Explicitly state: Disabling the data toggle mid-operation requires full block sync to recover missing historical data.
Are you willing to implement this feature?
Yes. Willing to lead development.
Questions for community discussion:
- Solution Selection: Do you agree with recommending Solution 2?
- Default Value: Should
storage.writeSectionBloomdefault totrue(data priority) orfalse(resource priority)?
Final Implementation:
Option 1 was chosen, which always writes section-bloom data.
Therefore, the node.jsonrpc.httpFullNodeEnable configuration option no longer controls section-bloom data writing; it will still be written even if it is disabled.
Similarly, blockbloom data is always written to the BlockCapsule object. This data is in-memory and not written to the block database. It is only used when filtering events using filters (eth_newFilter, eth_getFilterLogs, and eth_getFilterChanges). Using filters requires the node.jsonrpc.httpFullNodeEnable configuration option to be enabled, just as before.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
