6 releases (stable)

Uses old Rust 2015

2.2.0 Jan 26, 2022
2.1.0 Jan 29, 2020
2.0.0 Oct 9, 2016
1.0.1 Oct 9, 2016
0.1.0 Aug 31, 2015

#1114 in Text processing

Download history 24/week @ 2024-03-13 30/week @ 2024-03-20 39/week @ 2024-03-27 70/week @ 2024-04-03 18/week @ 2024-04-10 39/week @ 2024-04-17 34/week @ 2024-04-24 49/week @ 2024-05-01 30/week @ 2024-05-08 18/week @ 2024-05-15 15/week @ 2024-05-22 24/week @ 2024-05-29 14/week @ 2024-06-05 17/week @ 2024-06-12 20/week @ 2024-06-19 14/week @ 2024-06-26

68 downloads per month
Used in 3 crates

MIT license

14KB
194 lines

xhtmlchardet

Basic character set detection for XML and HTML in Rust.

Build Status Documentation Latest Version

Minimum Supported Rust Version: 1.24.0

Example

use std::io::Cursor;
extern crate xhtmlchardet;

let text = b"<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><channel><title>Example</title></channel>";
let mut text_cursor = Cursor::new(text.to_vec());
let detected_charsets: Vec<String> = xhtmlchardet::detect(&mut text_cursor, None).unwrap();
assert_eq!(detected_charsets, vec!["iso-8859-1".to_string()]);

Rationale

I wrote a feed crawler that needed to determine the character set of fetched content so that it could be normalised to UTF-8. Initially I used the uchardet crate but I encountered some situations where it misdetected the charset. I collected all these edge cases together and built a test suite. Then I implemented this crate, which passes all of those tests. It uses a fairly naïve approach derived from section F of the XML specification.

No runtime deps