Active direct dependers over time: total, gained/lost
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
2
2
4
4
4
4
4
4
4
5
5
5
5
5
6
6
7
7
10
11
12
13
13
14
15
31
32
31
38
44
43
42
52
49
53
51
48
51
54
57
57
58
2020 2021 2022 2023 2024 2025

Tokenizers is used at run time in 137 crates (of which 16 optionally). It is a direct run-time dependency in 90 crates. It's used only as a dev dependency in 5 crates (of which 4 directly).

Number of dependers Tokenizers version Downloads/month
65 0.21.1 46K
0 0.21.0 51K
12 0.20.4 18K
1 0.20.3 20K
20 0.19.1 21K
20 0.15.2 44K
9 0.14.1 4.5K
10 0.13.4 5.3K
4 0.11.3 0
1 0.10.1 270
Depender (with downloads and own rev deps) Tokenizers version
11K 1 toktrie_hf_tokenizers >=0.20.0, <1.0.0
31K 15 text-splitter optional ^0.21
5.3K 12 fastembed ^0.21
2.7K 2 gibberish-or-not ^0.14.1
3.7K 5 lancedb optional ^0.19.1
900 glowrs ^0.19.1
3.1K 19 rust-bert optional ^0.20
800 sbv2_core ^0.20.0
800 aleph-alpha-client ^0.21.0
750 gline-rs ^0.21.0
550 coze ^0.15.2
440 rai-models ^0.15.0
440 2 claude-tokenizer ^0.20.0
410 gpt_sovits_rs ^0.20.0
400 1 kalosm-llama ^0.21.0
390 4 candle-datasets ^0.21.0
380 4 candle-examples ^0.21.0
360 valentinus >=0.13.4
330 1 rwhisper ^0.21.0
290 1 rbert ^0.21.0
280 mpnet-rs ^0.15.1
260 embed_anything ^0.21.1
240 tkn-cli ^0.21.1
180 4 alith-models ^0.21.1
170 bundle_repo ^0.21
170 1 kalosm-ocr ^0.21.0
160 llms-from-scratch-rs ^0.21.0
150 1 rwuerstchen ^0.21.0
140 simple-whisper ^0.19.1
130 lsp-ai ^0.14.1
130 1 alith-inference ^0.21.1
120 gte-rs ^0.21.0
110 gliclass-rs ^0.21.0
rphi ^0.19.1
smartgpt ^0.13.3
rusty-dawg ^0.20.0
glimpse ^0.21.0
ct2rs ^0.21.0
ragtime ^0.19
oxidized-transformers ^0.15
reco-forge ^0.15.0
edge-transformers ^0.13
promptbox ^0.15.0
ug-llama ^0.20.3
usls ^0.15.2
modeling ^0.11.0
3 llm_models ^0.21.0
r2md ^0.21.0
enfer_core ^0.15.0
bm42 ^0.19
outlines-core =0.20.3
dataflow_nlp ^0.11
fashion-clip-rs ^0.15
musicgpt ^0.19.1
moz_genai ^0.15.0
aiochatgpt ^0.13.2
callm ^0.19
2 diffusion_rs_common ^0.21.0
bge ^0.15.2
4 sapiens ^0.14.1
spoilers ^0
1 clip_txt ^0.13.3
aleph-alpha-api ^0.15.0
cracken ~0.11.0
sbert ^0.11
1 stable-diffusion ^0.15.0
bridge-common ^0.19.1
moshi-backend ^0.15.2
rsllm ^0.19.1
candle_embed ^0.19.1
token-counter ^0.19.1
wimbd ^0.15.1
1 wonnx-preprocessing ^0.13.3
anonymize_rs ^0.13.3
candle-chat-types ^0.13.4
codegeex4-candle ^0.19.1
2 diffuse_rs_common ^0.21.0
1 diffuse_rs_core ^0.21.0
1 diffusion_rs_core ^0.21.0
heapswap ^0.15.1
inference_core ^0.15.0
prefrontal ^0.15
rmistral ^0.13.4
yolors ^0.15.0
260 code-splitter optional ^0.19.1
160 2 beet_ml optional ^0.19.1
1 natural-tts optional ^0.21.0
sentinellm_ai optional ^0.20
aleph-alpha-tokenizer optional ^0.10.1
norma optional ^0.20.0
461K 57 ort dev >=0.13.4
650 3 general-sam dev ^0.21.1
120 mtc-token-healing dev ^0.21.1
nano-vectordb-rs dev ^0.21.0