#html #css #selector #scraping

bin+lib scraper

HTML parsing and querying with CSS selectors

18 releases (10 breaking)

0.11.0 Sep 30, 2019
0.10.1 Jul 7, 2019
0.10.0 Apr 8, 2019
0.9.2 Mar 22, 2019
0.3.1 Mar 5, 2016

#18 in Web programming

Download history 645/week @ 2019-12-15 522/week @ 2019-12-22 490/week @ 2019-12-29 666/week @ 2020-01-05 1094/week @ 2020-01-12 1415/week @ 2020-01-19 1525/week @ 2020-01-26 1637/week @ 2020-02-02 1006/week @ 2020-02-09 921/week @ 2020-02-16 1399/week @ 2020-02-23 942/week @ 2020-03-01 1027/week @ 2020-03-08 1073/week @ 2020-03-15 1434/week @ 2020-03-22 972/week @ 2020-03-29

2,995 downloads per month
Used in 53 crates (45 directly)

ISC and MPL-2.0 licenses

47KB
1K SLoC

scraper

HTML parsing and querying with CSS selectors.

scraper is on Crates.io and GitHub.

Scraper provides an interface to Servo's html5ever and selectors crates, for browser-grade parsing and querying.

Examples

Parsing a document

use scraper::Html;

let html = r#"
    <!DOCTYPE html>
    <meta charset="utf-8">
    <title>Hello, world!</title>
    <h1 class="foo">Hello, <i>world!</i></h1>
"#;

let document = Html::parse_document(html);

Parsing a fragment

use scraper::Html;
let fragment = Html::parse_fragment("<h1>Hello, <i>world!</i></h1>");

Parsing a selector

use scraper::Selector;
let selector = Selector::parse("h1.foo").unwrap();

Selecting elements

use scraper::{Html, Selector};

let html = r#"
    <ul>
        <li>Foo</li>
        <li>Bar</li>
        <li>Baz</li>
    </ul>
"#;

let fragment = Html::parse_fragment(html);
let selector = Selector::parse("li").unwrap();

for element in fragment.select(&selector) {
    assert_eq!("li", element.value().name());
}

Selecting descendent elements

use scraper::{Html, Selector};

let html = r#"
    <ul>
        <li>Foo</li>
        <li>Bar</li>
        <li>Baz</li>
    </ul>
"#;

let fragment = Html::parse_fragment(html);
let ul_selector = Selector::parse("ul").unwrap();
let li_selector = Selector::parse("li").unwrap();

let ul = fragment.select(&ul_selector).next().unwrap();
for element in ul.select(&li_selector) {
    assert_eq!("li", element.value().name());
}

Accessing element attributes

use scraper::{Html, Selector};

let fragment = Html::parse_fragment(r#"<input name="foo" value="bar">"#);
let selector = Selector::parse(r#"input[name="foo"]"#).unwrap();

let input = fragment.select(&selector).next().unwrap();
assert_eq!(Some("bar"), input.value().attr("value"));

Serializing HTML and inner HTML

use scraper::{Html, Selector};

let fragment = Html::parse_fragment("<h1>Hello, <i>world!</i></h1>");
let selector = Selector::parse("h1").unwrap();

let h1 = fragment.select(&selector).next().unwrap();

assert_eq!("<h1>Hello, <i>world!</i></h1>", h1.html());
assert_eq!("Hello, <i>world!</i>", h1.inner_html());

Accessing descendent text

use scraper::{Html, Selector};

let fragment = Html::parse_fragment("<h1>Hello, <i>world!</i></h1>");
let selector = Selector::parse("h1").unwrap();

let h1 = fragment.select(&selector).next().unwrap();
let text = h1.text().collect::<Vec<_>>();

assert_eq!(vec!["Hello, ", "world!"], text);

License: ISC

Dependencies

~3MB
~63K SLoC