34 releases (7 breaking)

0.7.3 Oct 24, 2022
0.6.2 Oct 8, 2022
0.2.1 Jul 24, 2022
0.0.11 Jan 5, 2022
0.0.6 Dec 30, 2021

#1942 in Text processing

Download history 10/week @ 2024-06-16 3/week @ 2024-06-23 145/week @ 2024-06-30 6/week @ 2024-07-14 6/week @ 2024-07-21 6/week @ 2024-07-28 4/week @ 2024-08-04 1/week @ 2024-08-11 2/week @ 2024-09-08 72/week @ 2024-09-22 2/week @ 2024-09-29

76 downloads per month
Used in 4 crates (3 directly)

MIT license

145KB
2.5K SLoC

Morsels Language Package

This is a tokenizer crate for the morsels_indexer cli tool, not intended to be used directly.

Dependencies

~205–740KB
~12K SLoC