Lib.rs

›
#tokenizer #hugging-face #structured #outputs #toktrie #llguidance

toktrie_hf_tokenizers

Owned by Michał Moskal.

10 releases

new 0.6.27 Feb 18, 2025
0.6.26 Feb 14, 2025

#4 in #outputs

Download history 431/week @ 2025-02-04 421/week @ 2025-02-11

852 downloads per month

MIT license

80KB
2K SLoC

HuggingFace tokenizers library support for toktrie and llguidance

Dependencies

~14–27MB
~391K SLoC