12 breaking releases
0.14.0 | May 29, 2025 |
---|---|
0.12.0 | May 23, 2025 |
0.11.0 | Apr 28, 2024 |
0.10.0 | Oct 13, 2023 |
0.1.1 | Feb 13, 2019 |
#180 in Database interfaces
8,850 downloads per month
Used in 2 crates
350KB
117 lines
tantivy-jieba
An adapter that bridges between tantivy and jieba-rs.
Usage
Add dependency tantivy-jieba
to your Cargo.toml
.
Example
use tantivy::tokenizer::*;
let mut tokenizer = tantivy_jieba::JiebaTokenizer {};
let mut token_stream = tokenizer.token_stream("测试");
assert_eq!(token_stream.next().unwrap().text, "测试");
assert!(token_stream.next().is_none());
Register tantivy tokenizer
use tantivy::schema::Schema;
use tantivy::tokenizer::*;
use tantivy::Index;
let tokenizer = tantivy_jieba::JiebaTokenizer {};
let index = Index::create_in_ram(schema);
index.tokenizers()
.register("jieba", tokenizer);
See examples/mod.rs for detailed example.
License
Dependencies
~16MB
~125K SLoC