34 releases (7 breaking)
| 0.7.3 | Oct 24, 2022 |
|---|---|
| 0.6.2 | Oct 8, 2022 |
| 0.2.1 | Jul 24, 2022 |
| 0.0.11 | Jan 5, 2022 |
| 0.0.6 | Dec 30, 2021 |
#324 in Internationalization (i18n)
104 downloads per month
Used in 2 crates
(via morsels_indexer)
265KB
7K
SLoC
Morsels Language Package
This is a tokenizer crate for the morsels_indexer cli tool, not intended to be used directly.
Dependencies
~225–750KB
~12K SLoC