718 stable releases

new 2.37.27 May 14, 2025
2.36.120 Apr 28, 2025
2.36.7 Mar 31, 2025
2.23.3 Dec 31, 2024
0.1.3 Jul 24, 2024

#2031 in Command line utilities

Download history 3464/week @ 2025-01-22 1843/week @ 2025-01-29 2722/week @ 2025-02-05 1973/week @ 2025-02-12 3520/week @ 2025-02-19 1175/week @ 2025-02-26 2598/week @ 2025-03-05 2647/week @ 2025-03-12 779/week @ 2025-03-19 3591/week @ 2025-03-26 3360/week @ 2025-04-02 1211/week @ 2025-04-09 4650/week @ 2025-04-16 3708/week @ 2025-04-23 390/week @ 2025-04-30 9207/week @ 2025-05-07

18,255 downloads per month

MIT license

23KB
287 lines

spider_utils

Utilities to use to help with getting the most out of spider.

CSS Scraping

use spider::{
    hashbrown::HashMap,
    packages::scraper::Selector,
};
use spider_utils::{QueryCSSMap, QueryCSSSelectSet, build_selectors, css_query_select_map_streamed};

async fn css_query_selector_extract() {
    let map = QueryCSSMap::from([(
        "list",
        QueryCSSSelectSet::from([".list", ".sub-list"]),
    )]);
    let data = css_query_select_map_streamed(
        r#"<html>
            <body>
                <ul class="list"><li>First</li></ul>
                <ul class="sub-list"><li>Second</li></ul>
            </body>
        </html>"#,
        &build_selectors(map),
    )
    .await;

    println!("{:?}", data);
    // {"list": ["First", "Second"]}
}

Features

You can use the feature flag indexset to order the CSS scraping extraction order.

Dependencies

~10–25MB
~445K SLoC