6 releases (stable)

Uses old Rust 2015

2.2.0 Jan 26, 2022
2.1.0 Jan 29, 2020
2.0.0 Oct 9, 2016
1.0.1 Oct 9, 2016
0.1.0 Aug 31, 2015

#1430 in Text processing

Download history 23/week @ 2024-06-17 19/week @ 2024-06-24 8/week @ 2024-07-01 10/week @ 2024-07-08 19/week @ 2024-07-15 14/week @ 2024-07-22 17/week @ 2024-07-29 52/week @ 2024-08-05 19/week @ 2024-08-12 7/week @ 2024-08-19 32/week @ 2024-08-26 6/week @ 2024-09-02 16/week @ 2024-09-09 9/week @ 2024-09-16 55/week @ 2024-09-23 30/week @ 2024-09-30

111 downloads per month
Used in 3 crates

MIT license

14KB
194 lines

xhtmlchardet

Basic character set detection for XML and HTML in Rust.

Build Status Documentation Latest Version

Minimum Supported Rust Version: 1.24.0

Example

use std::io::Cursor;
extern crate xhtmlchardet;

let text = b"<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><channel><title>Example</title></channel>";
let mut text_cursor = Cursor::new(text.to_vec());
let detected_charsets: Vec<String> = xhtmlchardet::detect(&mut text_cursor, None).unwrap();
assert_eq!(detected_charsets, vec!["iso-8859-1".to_string()]);

Rationale

I wrote a feed crawler that needed to determine the character set of fetched content so that it could be normalised to UTF-8. Initially I used the uchardet crate but I encountered some situations where it misdetected the charset. I collected all these edge cases together and built a test suite. Then I implemented this crate, which passes all of those tests. It uses a fairly naïve approach derived from section F of the XML specification.

No runtime deps