#text-encoding #iconv #wasm #unicode #convert-text #winapi

iconv-native

A lightweight text encoding converter based on platform native API or libiconv

2 releases

0.1.0 Nov 16, 2024
0.1.0-alpha.1 Oct 7, 2024

#538 in Text processing

Download history 160/week @ 2024-10-06 18/week @ 2024-10-13 1/week @ 2024-10-20 6/week @ 2024-11-03 64/week @ 2024-11-10 55/week @ 2024-11-17 2/week @ 2024-11-24

127 downloads per month

MIT/Apache

71KB
1.5K SLoC

iconv-native-rs

crates.io Released API docs MIT licensed CI

A lightweight text encoding converter based on platform native APIs or libiconv.

Platforms

Windows

By default this crate uses MultiByteToWideChar and WideCharToMultiByte functions, controlled by feature win32. Since UTF-32 is not supported by these functions, widestring crate is used to convert UTF-32 to UTF-16 and vice versa.

You may also disable default features and enable libiconv to use the libiconv library instead.

Linux

On Linux with glibc, the built-in iconv is used by default, controlled by feature libc-iconv. You may also disable default features and enable libiconv to use the libiconv library instead.

Other libcs may not have an iconv implementation that is compatible with glibc's (specifically the //IGNORE and //TRANSLIT extensions and proper BOM handling), hence libc-iconv feature does not apply to them. By default, fallback-libiconv feature applies and will link to the libiconv library. Make sure to have libiconv installed on user's system.

macOS

Same as Linux with glibc. You may also disable default features and enable libiconv to use the libiconv library instead.

Web (WASM)

Uses TextDecoder and TextEncoder Web APIs. widestring crate is used to handle UTF-16 and UTF-32 related conversions.

[!IMPORTANT] As per Encoding Standard, a standard-compliant browser supports only UTF-8 when using TextEncoder, hence conversions to any encodings other than UTF-8/UTF-16/UTF-32 (including LE/BE variants) are not supported and will result in an UnknownConversion error. Consider import a polyfill and enable wasm-nonstandard-allow-legacy-encoding feature if full encoding support is required, in which case most of the encodings will work. However, there is no guarantee as it is not a standard-compliant behavior.

Conversions from legacy encodings are not affected by this limitation. See Encoding Standard for more details.

Other

On other platforms, the libiconv library is used by default, controlled by feature fallback-libiconv.

Dependencies

~0–10MB
~101K SLoC