#bitcoin #crypto #parser #block #tracking #optimized #data

bin+lib bitcoin-block-parser

Fast optimized parser for the bitcoin blocks data with UTXO tracking

11 unstable releases (3 breaking)

new 0.5.2 Nov 23, 2024
0.5.1 Nov 23, 2024
0.3.4 Nov 14, 2024
0.3.2 Oct 18, 2024
0.1.1 Sep 13, 2024

#752 in Magic Beans

Download history 152/week @ 2024-09-07 177/week @ 2024-09-14 80/week @ 2024-09-21 23/week @ 2024-09-28 2/week @ 2024-10-05 215/week @ 2024-10-12 145/week @ 2024-10-19 6/week @ 2024-10-26 13/week @ 2024-11-02 199/week @ 2024-11-09 36/week @ 2024-11-16

257 downloads per month

MIT license

56KB
912 lines

Bitcoin Block Parser

Crates.io Docs MIT licensed

Fast optimized parser for bitcoin blocks with input and output tracking.

⚠️ The API is still evolving and should not be considered stable until release 1.0.0

Features

  • Parses blocks into the Rust bitcoin Block format for easier manipulation
  • Can track if any TxOut is spent or unspent for calculations on the UTXO set
  • Can track the Amount of every TxIn for calculating metrics such as fee rates
  • Multithreaded in-memory parsing provides the fastest block parsing performance

Requirements / Benchmarks

  • You must be running a non-pruning bitcoin node (this is the default configuration)
  • We recommend using fast storage (e.g. NVMe) and a multithreaded CPU for best performance
  • See benchmarks below to understand how much RAM you may need:
Function Time Memory
BlockParser::parse()
Parses blocks
2m 55s 0.9 GB
UtxoParser::parse()
Tracks input amounts (no filter)
7m 13s 25.0 GB
UtxoParser::create_filter()
Creates new filter
16m 09s 5.6 GB
UtxoParser::load_filter().parse()
Tracks input & outputs (with filter)
7m 46s 11.8 GB

Our benchmarks were run on NVMe storage with a 32-thread processor on 800,000 blocks.

Quick Usage

See BlockParser for details on how to parse blocks:

use bitcoin_block_parser::*;

// Initialize a logger (if you want to monitor parsing progress)
env_logger::builder().filter_level(log::LevelFilter::Info).init();

// Parse all blocks in the directory and map them to total_size
let parser = BlockParser::new("/home/user/.bitcoin/blocks/").unwrap();
for size in parser.parse(|block| block.total_size()) {
  // Do something with the block sizes
}

See UtxoParser for details on how to track inputs and outputs:

use bitcoin_block_parser::*;

let parser = UtxoParser::new("/home/user/.bitcoin/blocks/").unwrap();
// Load a filter file or create a new one for tracking output status
let blocks = parser.load_or_create_filter("filter.bin").unwrap();
for txdata in blocks.parse(|block| block.txdata) {
  for tx in txdata {
    for (output, status) in tx.output() {
      // Do something with the output status
    }
    for (input, amount) in tx.input() {
      // Do something with the input amounts
    }
  }
}

Dependencies

~11–16MB
~178K SLoC