#list #int #array #performance #integer

group-varint-offset-encoding

Fast algorithm to compress lists if integers in blocks of 4. Appends 0s to blocks, if length of input is not divisable by 4.

2 releases

0.1.1 Nov 15, 2022
0.1.0 Nov 15, 2022

#542 in Compression

MIT/Apache

30KB
763 lines

Group Varint Offset Encoding

  • Compressed integers in blocks of 3.
  • Has good compretion rate, even with outliers.
  • Utilized offset encoding to store less bytes.
  • Appends zeros to your data, if it can't be grouped into 3. Manually keep track of the exact amount if needed.

Usage

let data: Vec<u32> = ...;

use group_varint_offset_encoding::{ compress, decompress };

// anything that can be iteratored into u32s works fine.
let compressed_data = compress(&data);

let decompressed_data = decompress(&compressed_data).to_vec();

Dependencies

~67KB