Lib.rs

› Algorithms
#tokenizer #castle #tokenize

castle_tokenizer

Owned by Albert Marashi.

50 releases (16 breaking)

0.20.2 Dec 30, 2022
0.19.0 Sep 28, 2022
0.9.4 Jul 5, 2022

#962 in Algorithms

Download history 4/week @ 2023-12-21 2/week @ 2023-12-28 6/week @ 2024-02-15 27/week @ 2024-02-22 12/week @ 2024-02-29 15/week @ 2024-03-07 11/week @ 2024-03-14 14/week @ 2024-03-21 23/week @ 2024-03-28

63 downloads per month
Used in 5 crates (4 directly)

MIT license

25KB
649 lines

Castle Tokenizer: tokenizer

Dependencies

~3.5–5.5MB
~97K SLoC