5 releases (1 stable)

✓ Uses Rust 2018 edition

new 1.0.0 Jun 23, 2020
0.3.0 Jul 28, 2019
0.2.1 Jul 10, 2019
0.2.0 Jul 7, 2019
0.1.0 Jul 6, 2019

#35 in Machine learning

Download history 8/week @ 2020-03-05 4/week @ 2020-03-12 16/week @ 2020-03-19 4/week @ 2020-03-26 1/week @ 2020-04-02 16/week @ 2020-04-09 12/week @ 2020-04-16 1/week @ 2020-04-23 1/week @ 2020-04-30 1/week @ 2020-05-14 2/week @ 2020-05-21 12/week @ 2020-05-28 57/week @ 2020-06-04 64/week @ 2020-06-11 82/week @ 2020-06-18

76 downloads per month

MIT license

5MB
66K SLoC

C++ 66K SLoC // 0.0% comments Rust 229 SLoC // 0.2% comments

Build Status Documentation Crate

BlingFire in Rust

blingfire is a thin Rust wrapper for the BlingFire tokenization library.

Add the library to Cargo.toml to get started

cargo add blingfire

The library exposes two functions text_to_words and text_to_sentences

use blingfire;

fn main() {
    let mut parsed = String::new();

    blingfire::text_to_words("Cat,sat on   the mat.", &mut parsed).unwrap();
    assert_eq!(parsed.as_str(), "Cat , sat on the mat .");

    blingfire::text_to_sentences("Cat sat. Dog barked.", &mut parsed).unwrap();
    assert_eq!(parsed.as_str(), "Cat sat.\nDog barked.");
}

The code is licensed under the MIT License.


lib.rs:

blingfire is a thin Rust wrapper for the BlingFire tokenization library.

Dependencies