Lib.rs

›
#tokenizer #castle #tokenize

castle_tokenizer

Owned by Albert Marashi.

50 releases (16 breaking)

0.20.2 Dec 30, 2022
0.19.0 Sep 28, 2022
0.9.4 Jul 5, 2022

#2 in #castle

Download history 7/week @ 2024-07-22 9/week @ 2024-07-29 7/week @ 2024-08-05 5/week @ 2024-08-12 32/week @ 2024-08-26 4/week @ 2024-09-02 3/week @ 2024-09-09 1/week @ 2024-09-16 51/week @ 2024-09-23 2/week @ 2024-09-30 1/week @ 2024-10-07 40/week @ 2024-10-14 25/week @ 2024-11-04

65 downloads per month
Used in 5 crates (4 directly)

MIT license

25KB
649 lines

Castle Tokenizer: tokenizer

Dependencies

~3.5–10MB
~99K SLoC