1 stable release

1.0.0 May 21, 2019

#1474 in Text processing

Download history 3234/week @ 2024-01-05 1952/week @ 2024-01-12 2690/week @ 2024-01-19 3444/week @ 2024-01-26 3641/week @ 2024-02-02 3505/week @ 2024-02-09 2739/week @ 2024-02-16 4354/week @ 2024-02-23 4510/week @ 2024-03-01 4331/week @ 2024-03-08 5685/week @ 2024-03-15 6386/week @ 2024-03-22 4944/week @ 2024-03-29 5782/week @ 2024-04-05 8169/week @ 2024-04-12 6295/week @ 2024-04-19

26,136 downloads per month
Used in 2 crates

MIT/Apache

14KB
270 lines

detone

docs.rs Apache 2 / MIT dual-licensed

An iterator adapter that takes an iterator over char yielding a sequence of chars in Normalization Form C (this precondition is not checked!) and yields chars either such that tone marks that wouldn't otherwise fit into windows-1258 are decomposed or such that text is decomposed into orthographic units.

Use cases include preprocessing before encoding Vietnamese text into windows-1258 or converting precomposed Vietnamese text into a form that looks like it was written with the (non-IME) Vietnamese keyboard layout (e.g. for machine learning training or benchmarking purposes).

Licensing

Please see the file named COPYRIGHT.

Documentation

Generated API documentation is available online.

Release Notes

1.0.0

  • Initial release.

No runtime deps