5 releases (3 breaking)

0.3.0 Oct 25, 2020
0.2.1 Jun 19, 2020
0.2.0 Jun 29, 2019
0.1.0 Jan 9, 2017
0.0.1 Apr 29, 2016

#134 in Text processing

Download history 26045/week @ 2024-06-14 22814/week @ 2024-06-21 25408/week @ 2024-06-28 22801/week @ 2024-07-05 19815/week @ 2024-07-12 23067/week @ 2024-07-19 25048/week @ 2024-07-26 20340/week @ 2024-08-02 37395/week @ 2024-08-09 25412/week @ 2024-08-16 36626/week @ 2024-08-23 28396/week @ 2024-08-30 31949/week @ 2024-09-06 19194/week @ 2024-09-13 30918/week @ 2024-09-20 31321/week @ 2024-09-27

119,032 downloads per month
Used in 656 crates (19 directly)

Apache-2.0

175KB
2.5K SLoC

Rust 2K SLoC // 0.1% comments Python 261 SLoC // 0.3% comments C++ 112 SLoC // 0.1% comments

xi-unicode

This crate contains unicode utilites adapted for working with non-contiguous bytes (such as a rope.)

Much of the contents of this repo are generated automatically by scripts from Unicode data files.

This current file is the result of some archaeology; documentation on how to rebuild the various files was missing, and I am attempting to reconstruct it.

data

Constructing the various tables require the various data files. These are available through the components of the Unicode standard directory, for a given unicode version. In particular, we require LineBreak.txt.

This file should be placed in a directory: I use data.

  • src/tables.rs is generated with the script located at tools/mk_tables.py, and can be built with,

    $ python3 tools/mk_tables.py data > src/tables.rs
    

    where data is the path to the created data directory.

  • the unit tests in src/lib.rs are also generated by this script, by passing the --tests and --tests-str flags (separately, to separate invocations) of the script, and then copying the output over into the body of these tests.

No runtime deps