34 releases (7 breaking)

0.7.3 Oct 24, 2022
0.6.2 Oct 8, 2022
0.2.1 Jul 24, 2022
0.0.11 Jan 5, 2022
0.0.6 Dec 30, 2021

#432 in Text processing

Download history 9/week @ 2022-11-24 11/week @ 2022-12-01 68/week @ 2022-12-08 11/week @ 2022-12-15 27/week @ 2022-12-22 22/week @ 2022-12-29 26/week @ 2023-01-05 24/week @ 2023-01-12 40/week @ 2023-01-19 31/week @ 2023-01-26 57/week @ 2023-02-02 62/week @ 2023-02-09 104/week @ 2023-02-16 106/week @ 2023-02-23 79/week @ 2023-03-02 16/week @ 2023-03-09

327 downloads per month
Used in 4 crates (3 directly)

MIT license

115KB
2K SLoC

Morsels Language Package

This is a tokenizer crate for the morsels_indexer cli tool, not intended to be used directly.

Dependencies

~210–475KB
~10K SLoC