#bitcoin #crypto #blocks #parser #tracking #optimized #data

bin+lib bitcoin-block-parser

Fast optimized parser for the bitcoin blocks data with UTXO tracking

8 releases

new 0.3.4 Nov 14, 2024
0.3.3 Nov 13, 2024
0.3.2 Oct 18, 2024
0.2.0 Sep 20, 2024
0.1.1 Sep 13, 2024

#321 in Magic Beans

Download history 248/week @ 2024-09-12 143/week @ 2024-09-19 36/week @ 2024-09-26 7/week @ 2024-10-03 2/week @ 2024-10-10 349/week @ 2024-10-17 15/week @ 2024-10-24 7/week @ 2024-10-31 58/week @ 2024-11-07

434 downloads per month

MIT license

48KB
739 lines

Bitcoin Block Parser

Crates.io Docs MIT licensed

Fast optimized parser for the bitcoin blocks data with UTXO tracking.

⚠️ The API is still evolving and should not be considered stable until release 1.0.0

Features

  • Parses blocks into the Rust bitcoin Block format for easier manipulation
  • Can track whether any TxOut in a Transaction is spent or unspent
  • Can track the Amount of every TxIn for calculating metrics such as fee rates
  • Or implement your own custom multithreaded BlockParser
  • Uses many optimizations to provide the best block parsing performance

Requirements / Benchmarks

  • You must be running a non-pruning bitcoin node (this is the default configuration)
  • You should look at the table below to understand how much RAM you need (tunable through Options)
  • We recommend using fast storage and a multithreaded CPU for best performance

Our benchmarks were run on NVMe storage with a 32-thread processor on 850,000 blocks:

Function Time Memory
DefaultParser 5 min 3.5 GB
InOrderParser 6 min 3.4 GB
FilterParser 17 min 9.3 GB
UtxoParser 39 min 17.5 GB

Quick Usage

  • To parse blocks pass in the blocks directory of your bitcoin node and call DefaultParser::parse_dir
  • If your algorithm requires the blocks to be processed in-order use InOrderParser::parse_dir
  • For advanced usage and implementing your own block parser see the blocks module docs
  • To track amounts through the transaction graph see the utxos module docs
use bitcoin_block_parser::*;

// Initialize a logger (if you want to monitor parsing progress)
env_logger::builder().filter_level(log::LevelFilter::Info).init();

// Iterates over all the blocks in the directory in parallel
// Use InOrderParser if your algorithm requires the blocks to be processed in-order
for block in DefaultParser.parse_dir("/home/user/.bitcoin/blocks").unwrap() {
  // Do whatever you want with the parsed block here
  block.unwrap().check_witness_commitment();
}

Dependencies

~8.5MB
~113K SLoC