#tokenizers #containing #rule-based

paltoquet

A library of rule-based general-purpose tokenizers

22 releases (10 breaking)

0.11.0 Nov 18, 2024
0.10.0 Sep 26, 2024
0.4.0 Jul 30, 2024

#484 in Text processing

Download history 129/week @ 2024-08-18 26/week @ 2024-08-25 25/week @ 2024-09-01 20/week @ 2024-09-08 304/week @ 2024-09-15 267/week @ 2024-09-22 80/week @ 2024-09-29 16/week @ 2024-10-06 20/week @ 2024-10-13 17/week @ 2024-10-20 11/week @ 2024-10-27 23/week @ 2024-11-03 132/week @ 2024-11-10 197/week @ 2024-11-17 38/week @ 2024-11-24 28/week @ 2024-12-01

395 downloads per month
Used in xan

MIT license

94KB
3K SLoC

Paltoquet

WIP...

Dependencies

~6MB
~142K SLoC