Lib.rs

›

#tokenize #tokenease #tokenizing

tokenease

WIP library to make tokenizing easy

Owned by JBourds.

Install
API reference
GitHub repo (jbourds)

1 unstable release

0.0.0	Dec 8, 2024

#48 in #tokenize

MIT license

2KB

tokenease

No runtime deps

See also: unscanny, tokenizers, charabia, sqlite3-parser, html5gum, erl_tokenize, notmecab, tinytoken, weatherkit-rs, actix-cookie, svm_merkle_tree

Lib.rs is an unofficial list of Rust/Cargo crates, created by kornelski. It contains data from multiple sources, including heuristics, and manually curated data. Content of this page is not necessarily endorsed by the authors of the crate. This site is not affiliated with nor endorsed by the Rust Project. If something is missing or incorrect, please file a bug.