Active direct dependers over time: total, gained/lost
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
2
2
4
4
4
4
4
4
4
5
5
5
5
5
6
6
7
7
10
11
12
13
13
14
15
31
32
31
38
44
43
42
52
49
53
51
48
51
54
57
57
59
2020 2021 2022 2023 2024 2025

Tokenizers is used at run time in 139 crates (of which 18 optionally). It is a direct run-time dependency in 92 crates. It's used only as a dev dependency in 5 crates (of which 4 directly).

Number of dependers Tokenizers version Downloads/month
65 0.21.1 60K
0 0.21.0 76K
12 0.20.4 43K
22 0.19.1 16K
21 0.15.2 21K
9 0.14.1 4.0K
10 0.13.4 5.0K
4 0.11.3 130
1 0.10.1 60
Depender (with downloads and own rev deps) Tokenizers version
32K 15 text-splitter optional ^0.21
7.3K 1 toktrie_hf_tokenizers >=0.20.0, <1.0.0
5.6K 12 fastembed ^0.21
1.7K rsllm ^0.19.1
4.1K 19 rust-bert optional ^0.20
1.0K aleph-alpha-client ^0.21.0
900 glowrs ^0.19.1
800 sbv2_core ^0.20.0
750 gline-rs ^0.21.0
700 2 gibberish-or-not ^0.14.1
2.5K 5 lancedb optional ^0.19.1
550 coze ^0.15.2
500 gpt_sovits_rs ^0.20.0
480 4 candle-examples ^0.21.0
440 rai-models ^0.15.0
430 r2md ^0.21.0
380 outlines-core =0.21.1
370 4 candle-datasets ^0.21.0
360 valentinus >=0.13.4
330 embed_anything ^0.21.1
320 2 claude-tokenizer ^0.20.0
280 mpnet-rs ^0.15.1
240 tkn-cli ^0.21.1
230 moshi-backend ^0.15.2
220 1 rbert ^0.21.0
220 candle_embed ^0.19.1
210 4 alith-models ^0.21.1
210 1 kalosm-llama ^0.21.0
190 1 rwhisper ^0.21.0
170 1 stable-diffusion ^0.15.0
130 simple-whisper ^0.19.1
130 ug-llama ^0.20.3
130 1 alith-inference ^0.21.1
130 lsp-ai ^0.14.1
120 gliclass-rs ^0.21.0
120 1 kalosm-ocr ^0.21.0
110 1 rwuerstchen ^0.21.0
110 vectordb-cli ^0.21.1
smartgpt ^0.13.3
rusty-dawg ^0.20.0
ragtime ^0.19
sbert ^0.11
usls ^0.15.2
bge ^0.15.2
oxidized-transformers ^0.15
reco-forge ^0.15.0
rphi ^0.19.1
gte-rs ^0.21.0
edge-transformers ^0.13
promptbox ^0.15.0
modeling ^0.11.0
glimpse ^0.21.0
ct2rs ^0.21.0
enfer_core ^0.15.0
3 llm_models ^0.21.0
dataflow_nlp ^0.11
fashion-clip-rs ^0.15
aiochatgpt ^0.13.2
bm42 ^0.19
moz_genai ^0.15.0
1 clip_txt ^0.13.3
musicgpt ^0.19.1
2 diffusion_rs_common ^0.21.0
spoilers ^0
bundle_repo ^0.21
4 sapiens ^0.14.1
callm ^0.19
cracken ~0.11.0
llms-from-scratch-rs ^0.21.0
token-counter ^0.19.1
wimbd ^0.15.1
1 wonnx-preprocessing ^0.13.3
aleph-alpha-api ^0.15.0
anonymize_rs ^0.13.3
bridge-common ^0.19.1
candle-chat-types ^0.13.4
codegeex4-candle ^0.19.1
2 diffuse_rs_common ^0.21.0
1 diffuse_rs_core ^0.21.0
1 diffusion_rs_core ^0.21.0
heapswap ^0.15.1
inference_core ^0.15.0
prefrontal ^0.15
rmistral ^0.13.4
yolors ^0.15.0
260 code-splitter optional ^0.19.1
180 norma optional ^0.20.0
160 1 natural-tts optional ^0.21.0
1 scirs2-text optional ^0.15
sentinellm_ai optional ^0.20
aleph-alpha-tokenizer optional ^0.10.1
2 beet_ml optional ^0.19.1
389K 59 ort dev >=0.13.4
600 3 general-sam dev ^0.21.1
120 mtc-token-healing dev ^0.21.1
nano-vectordb-rs dev ^0.21.0