Lib.rs

› Text processing › tergo
#language #tokenizer #tokenize

tergo-tokenizer

Owned by Konrad Pagacz.

3 unstable releases

0.2.1 Oct 27, 2024
0.2.0 Oct 27, 2024
0.1.0 Sep 14, 2024

#1628 in Text processing

Download history 155/week @ 2024-09-09 48/week @ 2024-09-16 16/week @ 2024-09-23 26/week @ 2024-09-30 158/week @ 2024-10-21 121/week @ 2024-10-28 97/week @ 2024-11-04 8/week @ 2024-11-11 57/week @ 2024-11-18 3/week @ 2024-11-25 9/week @ 2024-12-02 16/week @ 2024-12-09

86 downloads per month
Used in 4 crates (3 directly)

MIT license

26KB
645 lines

R language tokenizer

Dependencies

~270KB