simdnbt

Simdnbt is a very fast NBT serializer and deserializer.

It was originally made as a joke but it ended up being too good of a joke so it's actually a thing now.

Usage

cargo add simdnbt

Deserializing

For deserializing, you'll likely want either simdnbt::borrow::read or simdnbt::owned::read. The difference is that the "borrow" variant requires you to keep a reference to the original buffer, but is significantly faster.

use std::borrow::Cow;
use std::io::Cursor;

fn example(item_bytes: &[u8]) {
    let nbt = simdnbt::borrow::read(&mut Cursor::new(item_bytes))
        .unwrap()
        .unwrap();
    let skyblock_id: Cow<str> = nbt
        .list("i")
        .and_then(|i| i.compounds())
        .and_then(|i| i.first())
        .and_then(|i| i.compound("tag"))
        .and_then(|tag| tag.compound("ExtraAttributes"))
        .and_then(|ea| ea.string("id"))
        .map(|id| id.to_string_lossy())
        .unwrap_or_default();
}

Serializing

use simdnbt::owned::{BaseNbt, Nbt, NbtCompound, NbtTag};

let nbt = Nbt::Some(BaseNbt::new(
    "",
    NbtCompound::from_values(vec![
        ("key".into(), NbtTag::String("value".into())),
    ]),
));
let mut buffer = Vec::new();
nbt.write(&mut buffer);

Performance guide

Use the borrow variant of Nbt if possible, and avoid allocating unnecessarily (for example, keep strings as Cow<str> if you can).

If you're using the owned variant of Simdnbt, switching to a faster allocator like mimalloc may help a decent amount (it's ~20% faster on my machine). Setting RUSTFLAGS='-C target-cpu=native' when running your code may sometimes also help a little bit.

Implementation details

The "SIMD" part of the name is there as a reference to simdjson, and isn't usually critical to Simdnbt's decoding speed. Regardless, Simdnbt does actually make use of SIMD instructions for two things:

• swapping the endianness of int arrays.
• checking if a string is plain ascii for faster MUTF-8 to UTF-8 conversion.

Additionally, Simdnbt takes some shortcuts which usually aren't taken by other libraries:

• simdnbt::borrow requires a reference to the original data.
• it doesn't validate/decode MUTF-8 strings or integer arrays while parsing.
• compounds aren't sorted, so lookup always does a linear search.

Several ideas are borrowed from simdjson, notably the usage of a tape.

Benchmarks

Simdnbt is the fastest NBT parser in Rust.

Here's a benchmark comparing Simdnbt against a few of the other fastest NBT crates for decoding complex_player.dat:

And for writing complex_player.dat:

The tables above were made from the compare benchmark in this repo, with cargo bench 'compare/complex_player.dat/'.

Note that the benchmark is somewhat unfair, since Simdnbt takes a few shortcuts that other libraries don't. See the Implementation Details section above for more info.

Also keep in mind that if you run your own benchmark you'll get different numbers, but the speeds should be about the same relative to each other.