#tokenizer #tokenize #eson

eson-tokenizer

The eson's tokenizer implementation

1 unstable release

0.0.1 Sep 12, 2024

#38 in #tokenize

Apache-2.0

210KB
5.5K SLoC

tokenizer

The implementation of eson's tokenizer

Dependencies

~1.5MB
~25K SLoC