2 releases

Uses old Rust 2015

0.1.1 Jul 21, 2018
0.1.0 Nov 1, 2015

#156 in Profiling

Download history 247/week @ 2023-10-30 258/week @ 2023-11-06 332/week @ 2023-11-13 385/week @ 2023-11-20 259/week @ 2023-11-27 251/week @ 2023-12-04 416/week @ 2023-12-11 329/week @ 2023-12-18 386/week @ 2023-12-25 340/week @ 2024-01-01 329/week @ 2024-01-08 365/week @ 2024-01-15 271/week @ 2024-01-22 344/week @ 2024-01-29 369/week @ 2024-02-05 502/week @ 2024-02-12

1,506 downloads per month
Used in 2 crates

MIT license

36KB
178 lines

rust-tinysegmenter

Crates.io

Rust implementation of TinySegmenter, which is a compact Japanese tokenizer.

Install

Adding the following to the Cargo.toml in your project:

[dependencies]
tinysegmenter = "0.1"

and import using extern crate:

extern crate tinysegmenter;

Usage

let tokens = tinysegmenter::tokenize("私の名前は中野です");
println!("{}", &tokens.join("|")); // 私|の|名前|は|中野|です

License

Copyright (c) 2015 woxtu

Licensed under the MIT license.

Dependencies

~29KB