| 1 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 1 | 1 | 1 | 2 | 2 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 | 5 | 5 | 5 | 6 | 7 | 7 | 8 | 8 | 11 |
12
|
13
|
13
|
13
|
14
|
15
|
31
|
32
|
31
|
38
|
42
|
| 2020 | 2021 | 2022 | 2023 | 2024 | ||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Tokenizers is used at run time in 64 crates (of which 5 optionally). It is a direct run-time dependency in 54 crates. It's used only as a dev dependency in 4 crates (of which 3 directly).
| Number of dependers | Tokenizers version | Downloads/month |
|---|---|---|
| 8 | 0.19.1 | 11K |
| 24 | 0.15.2 | 40K |
| 9 | 0.14.1 | 5.7K |
| 22 | 0.13.4 | 38K |
| 4 | 0.11.3 | 140 |
| 1 | 0.10.1 | 40 |
| Depender (with downloads and own rev deps) | Tokenizers version | |||
|---|---|---|---|---|
| 5.4K | 4 | text-splitter | optional | ^0.19.1 |
| 1.5K | glowrs | ^0.19.1 | ||
| 1.2K | 1 | fastembed | ^0.14 | |
| 1.1K | rsllm | ^0.15.0 | ||
| 2.7K | 13 | rust-bert | optional | ^0.15 |
| 340 | ct2rs | ^0.19.1 | ||
| 310 | usls | ^0.15.2 | ||
| 290 | bge | ^0.15.2 | ||
| 240 | 3 | candle-datasets | ^0.15.0 | |
| 220 | candle_embed | ^0.19.0 | ||
| 180 | mpnet-rs | ^0.15.1 | ||
| 170 | 3 | candle-examples | ^0.15.0 | |
| 130 | coze | ^0.15.2 | ||
| 120 | 5 | kalosm-sample | ^0.13.4 | |
| 110 | 6 | kalosm-language-model | ^0.13.4 | |
| 1 | kalosm-ocr | ^0.13.4 | ||
| 1 | kalosm-sound | ^0.14.0 | ||
| 1 | stable-diffusion | ^0.15.0 | ||
| 1 | segment-anything-rs | ^0.13.4 | ||
| 1 | rwuerstchen | ^0.13.4 | ||
| oxidized-transformers | ^0.15 | |||
| cracken | ~0.11.0 | |||
| modeling | ^0.11.0 | |||
| edge-transformers | ^0.13 | |||
| enfer_core | ^0.15.0 | |||
| 1 | rphi | ^0.13.4 | ||
| spoilers | ^0 | |||
| 2 | rbert | ^0.13.4 | ||
| wimbd | ^0.15.1 | |||
| 4 | sapiens | ^0.14.1 | ||
| 1 | kalosm-language | ^0.13.4 | ||
| moz_genai | ^0.15.0 | |||
| 1 | rwhisper | ^0.14.0 | ||
| 2 | kalosm-llama | ^0.13.4 | ||
| fashion-clip-rs | ^0.15 | |||
| dataflow_nlp | ^0.11 | |||
| yolors | ^0.15.0 | |||
| inference_core | ^0.15.0 | |||
| 1 | clip_txt | ^0.13.3 | ||
| 1 | wonnx-preprocessing | ^0.13.3 | ||
| heapswap | ^0.15.1 | |||
| candle-chat-types | ^0.13.4 | |||
| sbert | ^0.11 | |||
| aiochatgpt | ^0.13.2 | |||
| promptbox | ^0.15.0 | |||
| rmistral | ^0.13.4 | |||
| aleph-alpha-api | ^0.15.0 | |||
| anonymize_rs | ^0.13.3 | |||
| smartgpt | ^0.13.3 | |||
| bridge-common | ^0.19.1 | |||
| reco-forge | ^0.15.0 | |||
| aleph-alpha-tokenizer | optional | ^0.10.1 | ||
| pyke-diffusers | optional | ^0.13 | ||
| 21K | 25 | ort | dev | >=0.13.4 |
| 380 | 1 | general-sam | dev | ^0.15.2 |
| kalosm | dev | ^0.15.0 | ||
| mtc-token-healing | dev | ^0.19.1 | ||