Active direct dependers over time: total, gained/lost
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
2
2
4
4
4
4
4
4
4
5
5
5
5
5
6
6
7
7
10
11
12
13
13
14
15
31
32
31
38
44
43
42
52
49
53
51
48
51
54
56
2020 2021 2022 2023 2024

Tokenizers is used at run time in 117 crates (of which 14 optionally). It is a direct run-time dependency in 82 crates. It's used only as a dev dependency in 5 crates (of which 4 directly).

Number of dependers Tokenizers version Downloads/month
42 0.21.0 55K
13 0.20.4 24K
1 0.20.3 13K
24 0.19.1 28K
22 0.15.2 30K
6 0.14.1 3.9K
9 0.13.4 5.1K
4 0.11.3 0
1 0.10.1 72
Depender (with downloads and own rev deps) Tokenizers version
26K 14 text-splitter optional ^0.21
4.3K 11 fastembed ^0.21
1.8K valentinus >=0.13.4
1.7K rsllm ^0.19.1
1.6K aleph-alpha-client ^0.21.0
4.1K 6 lancedb optional ^0.19.1
850 gpt_sovits_rs ^0.20.0
850 toktrie_hf_tokenizers >=0.20.0, <1.0.0
800 sbv2_core ^0.20.0
700 outlines-core =0.20.3
2.6K 17 rust-bert optional ^0.20
550 coze ^0.15.2
440 rai-models ^0.15.0
420 musicgpt ^0.19.1
410 embed_anything ^0.20.2
410 1 rbert ^0.21.0
390 2 claude-tokenizer ^0.20.0
340 4 candle-datasets ^0.19.1
340 1 kalosm-llama ^0.21.0
320 4 candle-examples ^0.19.1
280 mpnet-rs ^0.15.1
280 1 rwhisper ^0.21.0
280 1 rwuerstchen ^0.21.0
260 1 kalosm-ocr ^0.21.0
220 rphi ^0.19.1
220 candle_embed ^0.19.1
190 3 llm_models ^0.21.0
190 lsp-ai ^0.14.1
190 gline-rs ^0.21.0
140 bundle_repo ^0.21
140 llms-from-scratch-rs ^0.21.0
130 gte-rs ^0.21.0
110 2 beet_ml ^0.19.1
110 moshi-backend ^0.15.2
110 4 sapiens ^0.14.1
110 prefrontal ^0.15
smartgpt ^0.13.3
sbert ^0.11
reco-forge ^0.15.0
edge-transformers ^0.13
promptbox ^0.15.0
fashion-clip-rs ^0.15
modeling ^0.11.0
1 wonnx-preprocessing ^0.13.3
ct2rs ^0.21.0
usls ^0.15.2
enfer_core ^0.15.0
dataflow_nlp ^0.11
aiochatgpt ^0.13.2
2 diffusion_rs_common ^0.21.0
glimpse ^0.21.0
heapswap ^0.15.1
callm ^0.19
1 diffusion_rs_core ^0.21.0
moz_genai ^0.15.0
bm42 ^0.19
cracken ~0.11.0
spoilers ^0
ragtime ^0.19
simple-whisper ^0.19.1
wimbd ^0.15.1
glowrs ^0.19.1
oxidized-transformers ^0.15
rusty-dawg ^0.20.0
bge ^0.15.2
1 clip_txt ^0.13.3
codegeex4-candle ^0.19.1
1 stable-diffusion ^0.15.0
aleph-alpha-api ^0.15.0
anonymize_rs ^0.13.3
bridge-common ^0.19.1
candle-chat-types ^0.13.4
2 diffuse_rs_common ^0.21.0
1 diffuse_rs_core ^0.21.0
inference_core ^0.15.0
rmistral ^0.13.4
token-counter ^0.19.1
yolors ^0.15.0
180 norma optional ^0.20.0
170 sentinellm_ai optional ^0.20
aleph-alpha-tokenizer optional ^0.10.1
code-splitter optional ^0.19.1
182K 48 ort dev >=0.13.4
290 nano-vectordb-rs dev ^0.21.0
190 mtc-token-healing dev ^0.20.1
180 3 general-sam dev ^0.15.2