Lib.rs

› Text processing › tergo
#language #tokenizer #tokenize

tergo-tokenizer

Owned by Konrad Pagacz.

3 unstable releases

0.2.1 Oct 27, 2024
0.2.0 Oct 27, 2024
0.1.0 Sep 14, 2024

#1574 in Text processing

Download history 193/week @ 2024-09-13 23/week @ 2024-09-20 24/week @ 2024-09-27 5/week @ 2024-10-04 2/week @ 2024-10-18 253/week @ 2024-10-25 47/week @ 2024-11-01 79/week @ 2024-11-08 14/week @ 2024-11-15 47/week @ 2024-11-22 9/week @ 2024-11-29 15/week @ 2024-12-06 3/week @ 2024-12-13

77 downloads per month
Used in 4 crates (3 directly)

MIT license

26KB
645 lines

R language tokenizer

Dependencies

~275KB