#chinese #dictionary #builder #morphological #cc-cedict #tokenize

lindera-cc-cedict-builder

A Chinese morphological dictionary builder for CC-CEDICT

38 releases (20 breaking)

new 0.31.0 May 28, 2024
0.29.0 Mar 18, 2024
0.27.2 Dec 30, 2023
0.27.1 Aug 25, 2023
0.12.2 Mar 23, 2022

#1674 in Text processing

Download history 4356/week @ 2024-02-04 4717/week @ 2024-02-11 5414/week @ 2024-02-18 5497/week @ 2024-02-25 5989/week @ 2024-03-03 5261/week @ 2024-03-10 7046/week @ 2024-03-17 5227/week @ 2024-03-24 5655/week @ 2024-03-31 5143/week @ 2024-04-07 5053/week @ 2024-04-14 4579/week @ 2024-04-21 4944/week @ 2024-04-28 5112/week @ 2024-05-05 4879/week @ 2024-05-12 5718/week @ 2024-05-19

21,104 downloads per month
Used in 24 crates (4 directly)

MIT license

82KB
2K SLoC

Lindera CC-CEDICT Builder

License: MIT Join the chat at https://gitter.im/lindera-morphology/lindera Crates.io

CC-CEDICT dictionary builder for Lindera.

Dictionary format

Refer to the manual for details on the unidic-mecab dictionary format and part-of-speech tags.

Index Name (Chinese) Name (English) Notes
0 表面形式 Surface
1 左语境ID Left context ID
2 右语境ID Right context ID
3 成本 Cost
4 词类 Major POS classification
5 词类1 Middle POS classification
6 词类2 Small POS classification
7 词类3 Fine POS classification
8 併音 pinyin
9 繁体字 traditional
10 簡体字 simplified
11 定义 definition

User dictionary format (CSV)

Simple version

Index Name (Japanese) Name (English) Notes
0 表面形式 Surface
1 词类 Major POS classification
2 併音 pinyin

Detailed version

Index Name (Japanese) Name (English) Notes
0 表面形式 Surface
1 左语境ID Left context ID
2 右语境ID Right context ID
3 成本 Cost
4 词类 POS
5 词类1 POS subcategory 1
6 词类2 POS subcategory 2
7 词类3 POS subcategory 3
8 併音 pinyin
9 繁体字 traditional
10 簡体字 simplified
11 定义 definition
12 - - After 12, it can be freely expanded.

How to use CC-CEDICT dictionary

For more details about lindera command, please refer to the following URL:

API reference

The API reference is available. Please see following URL:

Dependencies

~9.5MB
~213K SLoC