46 releases (26 breaking)

0.34.0 Apr 8, 2024
0.33.0 Mar 18, 2024
0.32.0 Feb 26, 2024
0.29.0 Dec 12, 2023
0.8.0-alpha.5 Mar 24, 2020

#994 in Magic Beans

Download history 1228/week @ 2024-01-02 1868/week @ 2024-01-09 1465/week @ 2024-01-16 1407/week @ 2024-01-23 1018/week @ 2024-01-30 1686/week @ 2024-02-06 2387/week @ 2024-02-13 2226/week @ 2024-02-20 2632/week @ 2024-02-27 2233/week @ 2024-03-05 1789/week @ 2024-03-12 2859/week @ 2024-03-19 2452/week @ 2024-03-26 2808/week @ 2024-04-02 2391/week @ 2024-04-09 1996/week @ 2024-04-16

10,185 downloads per month
Used in 79 crates (via sc-client-db)

GPL-3.0-or-later…

405KB
9K SLoC

State database maintenance. Handles canonicalization and pruning in the database. The input to this module is a ChangeSet which is basically a list of key-value pairs (trie nodes) that were added or deleted during block execution.

Canonicalization

Canonicalization window tracks a tree of blocks identified by header hash. The in-memory overlay allows to get any node that was inserted in any of the blocks within the window. The tree is journaled to the backing database and rebuilt on startup. Canonicalization function selects one root from the top of the tree and discards all other roots and their subtrees.

Pruning

See RefWindow for pruning algorithm details. StateDb prunes on each canonicalization until pruning constraints are satisfied.

License: GPL-3.0-or-later WITH Classpath-exception-2.0


lib.rs:

State database maintenance. Handles canonicalization and pruning in the database.

Canonicalization.

Canonicalization window tracks a tree of blocks identified by header hash. The in-memory overlay allows to get any trie node that was inserted in any of the blocks within the window. The overlay is journaled to the backing database and rebuilt on startup. There's a limit of 32 blocks that may have the same block number in the canonicalization window.

Canonicalization function selects one root from the top of the tree and discards all other roots and their subtrees. Upon canonicalization all trie nodes that were inserted in the block are added to the backing DB and block tracking is moved to the pruning window, where no forks are allowed.

Canonicalization vs Finality

Database engine uses a notion of canonicality, rather then finality. A canonical block may not be yet finalized from the perspective of the consensus engine, but it still can't be reverted in the database. Most of the time during normal operation last canonical block is the same as last finalized. However if finality stall for a long duration for some reason, there's only a certain number of blocks that can fit in the non-canonical overlay, so canonicalization of an unfinalized block may be forced.

Pruning.

See RefWindow for pruning algorithm details. StateDb prunes on each canonicalization until pruning constraints are satisfied.

Dependencies

~12–21MB
~318K SLoC