1 unstable release
0.1.0 | Jan 12, 2023 |
---|
#8 in #block-index
Used in summavy
71KB
1.5K
SLoC
SSTable
The tantivy-sstable
crate is yet another sstable crate.
It has been designed to be used in quickwit
:
- as an alternative to the default tantivy fst dictionary.
- as a way to store the column index for dynamic fast fields.
The benefit compared to the fst crate is locality. Searching a key in the fst crate requires downloading the entire dictionary.
Once the sstable index is downloaded, running a get
in the sstable
crate only requires a single fetch.
Right now, the block index and the default block size have been thought for quickwit, and the performance of a get is very bad.
Sorted strings?
SSTable stands for Sorted String Table. Strings have to be insert in sorted order.
That sorted order is used in different ways:
- it makes gets and streaming ranges of keys possible.
- it allows incremental encoding of the keys
- the front compression is leveraged to optimize the intersection with an automaton
Dependencies
~5MB
~91K SLoC