#sqlite-extension #tokenizer #extension #sqlite #fts5

libsimple

Rust bindings to simple, a SQLite3 fts5 tokenizer which supports Chinese and PinYin

2 unstable releases

0.2.1 Apr 16, 2024
0.2.0 Apr 16, 2024
0.1.0 Apr 15, 2024

#884 in Database interfaces

Download history 356/week @ 2024-04-15

356 downloads per month

MIT license

13MB
180K SLoC

C 178K SLoC // 0.2% comments C++ 1K SLoC // 0.1% comments Rust 110 SLoC

libsimple

Crate GitHub last commit GitHub issues GitHub pull requests GitHub

Description

Rust bindings to simple, a SQLite3 fts5 tokenizer which supports Chinese and PinYin.

Usage

Add this to your Cargo.toml:

[dependencies]
libsimple = "~0.2"

Example

use anyhow::Result;
use tempfile::tempdir;

fn main() -> Result<()> {
    libsimple::enable_auto_extension()?;
    let dir = tempdir()?;
    libsimple::release_dict(&dir)?;
    
    let conn = rusqlite::Connection::open_in_memory()?;
    libsimple::set_dict(&conn, &dir)?;
    conn.execute_batch("
        CREATE VIRTUAL TABLE d USING fts5(id, text, tokenize = 'simple');
        INSERT INTO d (id, text) VALUES (1, '中华人民共和国国歌');
        INSERT INTO d (id, text) VALUES (2, '周杰伦');
    ")?;
    assert_eq!(1, conn
        .query_row("SELECT id FROM d WHERE text MATCH jieba_query('中华国歌')", [], |row| row.get::<_, i64>(0))?
    );
    assert_eq!(2, conn
        .query_row("SELECT id FROM d WHERE text MATCH simple_query('zhoujiel')", [], |row| row.get::<_, i64>(0))?
    );
    Ok(())
}

License

Licensed under MIT license (LICENSE or http://opensource.org/licenses/MIT)

Dependencies

~22MB
~412K SLoC