indexkv

A performance-focused, persistent, async, key-value store

28 releases

0.7.4 Feb 2, 2023
0.6.2 Jan 4, 2023
0.6.0 Jan 17, 2022
0.5.1 Dec 23, 2021
0.2.6 Mar 7, 2021


Used in imdb-async

MIT license

41KB
862 lines

indexkv

Provides [Store], an on-disk storage bucket. Conceptually, this is not dissimilar to a map<u64, T> that provides fast (by persistent storage standards) key lookups, low memory complexity, and fast sequential writes at the expense of random writes; random writes are unsupported. See Store::write for details on writes.

While reading concurrently is safe, reading while a write is ongoing is a footgun.

Examples

Automatically keying values with Iterator::enumerate:

use indexkv::Store;

#[tokio::main]
async fn main() {
   	let path = tempfile::NamedTempFile::new().unwrap().into_temp_path();
   	let mut store: Store<u8, String> = Store::new(path).await.unwrap();

   	let stream = futures::stream::iter(
   		["zero", "one", "two", "three", "four", "five"]
   			.into_iter()
   			.enumerate()
   			.map(|(i, v)| (i as u8, v.to_string()))
   	);
   	store.write_infallible(stream, 0).await.unwrap();

   	let result = store.get(2).await.unwrap();
   	assert_eq!(result, "two");
   	let result = store.get_many(&[0, 4]).await.unwrap();
   	assert_eq!(result.len(), 2);
   	assert_eq!(result.get(&0), Some(&"zero".to_string()));
   	assert_eq!(result.get(&2), None);
   	assert_eq!(result.get(&4), Some(&"four".to_string()));
}

Manually keying values with an external "ID":

use indexkv::{Error, Store};

#[tokio::main]
async fn main() {
   	let path = tempfile::NamedTempFile::new().unwrap().into_temp_path();
   	let mut store: Store<u16, String> = Store::new(path).await.unwrap();

   	let stream = futures::stream::iter([
   		(200, "two hundred".to_string()),
   		(1337, "thirteen hundred thirty-seven".to_string()),
   		(75, "seventy-five".to_string()),
   		(20, "twenty".to_string())
   	]);
   	store.write_infallible(stream, 4).await.unwrap();

   	let result = store.get(2).await;
   	assert!(matches!(result, Err(Error::NotFound(2))));
   	let result = store.get(75).await.unwrap();
   	assert_eq!(result, "seventy-five".to_string());
}

Dependencies

~4–10MB
~99K SLoC