#chinese #traditional #convert #hanzi #simplified

character_converter

Turn Traditional Chinese script ot Simplified Chinese script and vice-versa and tokenize

11 releases (7 stable)

2.1.4 Oct 12, 2022
2.1.3 Sep 10, 2022
2.1.2 Aug 2, 2022
2.1.1 Jul 16, 2022
0.1.6 May 13, 2020

#240 in Text processing

Download history 1656/week @ 2022-12-02 1706/week @ 2022-12-09 1592/week @ 2022-12-16 1507/week @ 2022-12-23 1335/week @ 2022-12-30 1152/week @ 2023-01-06 900/week @ 2023-01-13 804/week @ 2023-01-20 794/week @ 2023-01-27 470/week @ 2023-02-03 352/week @ 2023-02-10 295/week @ 2023-02-17 194/week @ 2023-02-24 162/week @ 2023-03-03 164/week @ 2023-03-10 108/week @ 2023-03-17

643 downloads per month
Used in 2 crates

MIT license

2.5MB
274 lines

character_converter


About

Turn Traditional Chinese script to Simplified Chinese script and vice-versa. Check string script to determine if string is Traditional or Simplified Chinese Characters.

This package also includes a largest first matching tokenizer.

Usage

extern crate character_converter;

use character_converter::{is_traditional, is_simplified, traditional_to_simplified, simplified_to_traditional, tokenize};

let traditional_text = "歐洲";
let simplified_text = "欧洲";

// Check script
assert!(is_traditional(traditional_text));

assert!(!is_simplified(traditional_text));

// Convert script
let result_three = traditional_to_simplified(traditional_text);
assert_eq!(result_three, simplified_text);

let result_four = simplified_to_traditional(simplified_text);
assert_eq!(result_four, traditional_text);

// Tokenize
let string = "好好学习天天向上.";
let tokens = vec!["好好", "学习", "天天", "向上"];
assert_eq!(tokens, tokenize(string));

Benchmarks

Run benchmarks using the nightly bench feature:

cargo +nightly bench --features=bench

License

MIT

Dependencies

~2MB
~17K SLoC