#spell-checking #case #tokenization #binary #box #divvunspell #zhfst-bhfst

app divvunspell-bin

Spellchecker for ZHFST/BHFST spellers, with case handling and tokenization support

2 releases (1 stable)

1.0.0 Dec 13, 2023
1.0.0-beta.1 Apr 20, 2021

#489 in Text processing

43 downloads per month

GPL-3.0 license

5.5K SLoC


An implementation of hfst-ospell in Rust, with added features like tokenization, case handling, and parallelisation.


Building and installing commandline tools

# For the `divvunspell` binary:
cargo install divvunspell-bin

# For `thfst-tools` binary:
cargo install thfst-tools

# To build the development version from this source, cd into the relevant directory and:
cargo install --path .

Building with gpt2 support on macOS aarch64

Clone this repo then:

brew install libtorch
LIBTORCH=/opt/homebrew/opt/libtorch cargo build --features gpt2 --bin divvunspell

No Rust?

curl https://sh.rustup.rs -sSf | sh
source $HOME/.cargo/env
rustup default stable
cargo build --release



divvunspell 0.5.0
Testing frontend for the Divvunspell library

    divvunspell [FLAGS] [OPTIONS] <--zhfst <ZHFST>|--bhfst <BHFST>|--acceptor <acceptor>> [WORDS]...

    -S, --always-suggest    Always show suggestions even if word is correct (implies -s)
    -h, --help              Prints help information
        --json              Output results in JSON
    -s, --suggest           Show suggestions for given word(s)
    -V, --version           Prints version information

        --acceptor <acceptor>    Use the given acceptor file
    -b, --bhfst <BHFST>          Use the given BHFST file
        --errmodel <errmodel>    Use the given errmodel file
    -n, --nbest <nbest>          Maximum number of results for suggestions
    -w, --weight <weight>        Maximum weight limit for suggestions
    -z, --zhfst <ZHFST>          Use the given ZHFST file

    <WORDS>...    The words to be processed



divvunspell-accuracy 1.0.0-alpha.5
Accuracy testing for Divvunspell.

    accuracy [OPTIONS] [ARGS]

    -h, --help       Prints help information
    -V, --version    Prints version information

    -c <config>             Provide JSON config file to override test defaults
    -o <JSON-OUTPUT>        The file path for the JSON report output
    -w <max-words>          Truncate typos list to max number of words specified

    <WORDS>    The 'input -> expected' list in tab-delimited value file (TSV)
    <ZHFST>    Use the given ZHFST file


Convert hfst and zhfst files to thfst and bhfst formats.

  • thfst: byte-aligned hfst for fast and efficient loading and memory mapping, required to run divvunspell on ARM processors
  • bhfst: thfst files wrapped in a box container; in the case of zhfst files converted to bhfst, the metadata file (index.xml in the zhfst archive) is converted to a json file for faster and leaner processing by the divvunspell library.


thfst-tools 1.0.0-alpha.5
Tromsø-Helsinki Finite State Transducer toolkit.

    thfst-tools <SUBCOMMAND>

    -h, --help       Prints help information
    -V, --version    Prints version information

    bhfst-info         Print metadata for BHFST
    help               Prints this message or the help of the given subcommand(s)
    hfst-to-thfst      Convert an HFST file to THFST
    thfsts-to-bhfst    Convert a THFST acceptor/errmodel pair to BHFST
    zhfst-to-bhfst     Convert a ZHFST file to BHFST

Speller testing

There's a prototype-level testing tool in support/accuracy-viewer. Use it like:

accuracy -o support/accuracy-viewer/public/report.json typos.txt sma.zhfst
cd support/accuracy-viewer
npm i && npm run dev

View in http://localhost:5000.

typos.txt is a TSV file with typos in the first column and expected correction in the second. More info by accuracy --help.


The crate divvunspell is licensed under either of

at your option.

The divvunspell, thfst-tools and accuracy binaries are licensed under the GPL version 3 license.


~510K SLoC