#string #unicode #validation #simd #utf-8 #unicode-characters

no-std simdutf

Unicode validation and transcoding at billions of characters per second

17 releases

0.5.1 Sep 1, 2024
0.4.18 Aug 11, 2024
0.4.17 May 2, 2024
0.4.16 Jan 1, 2024
0.3.0 Jul 27, 2022

#405 in Text processing

MIT license

1.5MB
30K SLoC

C++ 29K SLoC // 0.2% comments Rust 718 SLoC // 0.0% comments

simdutf

Latest Version Documentation License

Unicode validation and transcoding at billions of characters per second.

This crate is the Rust binding of simdutf.

Documentation: https://docs.rs/simdutf


lib.rs:

Unicode validation and transcoding at billions of characters per second.

This crate is the Rust binding of simdutf.

Compilation

This crate works out of the box as long as you have a C++11-compatible toolchain installed correctly.

simdutf links C++ standard library, which adds a dynamic linking dependency.

For more details, see simdutf documentation and cc documentation.

Here is an example for local benchmark:

export RUSTFLAGS='-C target-cpu=native'
export CXXFLAGS='-march=native'
cargo build --release

Dependencies