#index #b-tree #index-file #temporary-files #key-value #dataset #transient

transient-btree-index

This crate allows you to create a BTree index backed by temporary files

6 releases (breaking)

0.5.1 Nov 14, 2023
0.5.0 Jul 19, 2022
0.4.0 Jul 17, 2022
0.3.0 Mar 10, 2022
0.1.0 Feb 10, 2022

#4 in #index-file

Download history 65/week @ 2024-06-13 66/week @ 2024-06-20 45/week @ 2024-06-27 62/week @ 2024-07-04 96/week @ 2024-07-11 63/week @ 2024-07-18 83/week @ 2024-07-25 53/week @ 2024-08-01 64/week @ 2024-08-08 107/week @ 2024-08-15 78/week @ 2024-08-22 100/week @ 2024-08-29 149/week @ 2024-09-05 119/week @ 2024-09-12 128/week @ 2024-09-19 87/week @ 2024-09-26

492 downloads per month
Used in 9 crates (5 directly)

Apache-2.0

81KB
1.5K SLoC

docs.rs Crates.io

Transient Index using B-Trees

transient-btree-index allows you to create a BTree index backed by temporary files. This is helpful if you

  • need to index large datasets (thus only working on disk) by inserting entries in unsorted order,
  • want to query entries (get and range queries) while the index is still constructed, e.g. to check existence of a previous entry, and
  • need support for all serde-serializable key and value types with varying key-size.

Because of its intended use case, it is therefore not possible to

  • delete entries once they are inserted (you can use Option values and set them to Option::None, but this will not reclaim any used space),
  • persist the index to a file (you can use other crates like sstable to create immutable maps), or
  • load an existing index file (you might want to use an immutable map file and this index can act as an "overlay" for all changed entries).

Example

use transient_btree_index::{BtreeConfig, BtreeIndex, Error};

fn main() -> std::result::Result<(), Error> {
    let mut b = BtreeIndex::<u16,u16>::with_capacity(BtreeConfig::default(), 10)?;
    b.insert(1,2)?;
    b.insert(200, 4)?;
    b.insert(20, 3)?;

    assert_eq!(true, b.contains_key(&200)?);
    assert_eq!(false, b.contains_key(&2)?);  

    assert_eq!(3, b.get(&20)?.unwrap());  

    for e in b.range(1..30)? {
        let (k, v) = e?;
        dbg!(k, v);
    }
    Ok(())
}

Dependencies

~3–12MB
~152K SLoC