3 releases
| 0.1.2 | Sep 28, 2023 |
|---|---|
| 0.1.1 | Sep 23, 2023 |
| 0.1.0 | Sep 20, 2023 |
#7 in #tokenizers
Used in retoken
20KB
520 lines
Retoken
Build &str tokenizers using regexes and token streams for a proc-macro2 / syn like experience when working with &str data.
Dependencies
~0.6–1MB
~23K SLoC