Tokenizers is used at run time in 8 crates (of which 2 optionally, 7 directly).
| Number of dependers | Tokenizers version | Downloads/month |
|---|---|---|
| 1 | 0.13.2 | 9.0K |
| 5 | 0.11.3 | 2.3K |
| 2 | 0.10.1 | 100 |
| Depender (with downloads and own rev deps) | Tokenizers version | |||
|---|---|---|---|---|
| modeling | ^0.11.0 | |||
| cracken | ~0.11.0 | |||
| sbert | ^0.10 | |||
| 1 | wonnx-preprocessing | ^0.11.3 | ||
| dataflow_nlp | ^0.11 | |||
| pyke-diffusers | optional | ^0.13 | ||
| aleph-alpha-tokenizer | optional | ^0.10.1 | ||