49 stable releases
new 2.5.2 | Sep 15, 2024 |
---|---|
2.5.1 | Sep 14, 2024 |
2.2.8 | Aug 31, 2024 |
0.2.3 | Aug 20, 2024 |
0.1.3 | Jul 24, 2024 |
#1935 in Web programming
6,537 downloads per month
610KB
12K
SLoC
spider_utils
Utilities to use to help with getting the most out of spider.
CSS Scraping
use spider::{
hashbrown::HashMap,
packages::scraper::Selector,
};
use spider_utils::{QueryCSSMap, QueryCSSSelectSet, build_selectors, css_query_select_map_streamed};
async fn css_query_selector_extract() {
let map = QueryCSSMap::from([(
"list",
QueryCSSSelectSet::from([".list", ".sub-list"]),
)]);
let data = css_query_select_map_streamed(
r#"<html>
<body>
<ul class="list"><li>First</li></ul>
<ul class="sub-list"><li>Second</li></ul>
</body>
</html>"#,
&build_selectors(map),
)
.await;
println!("{:?}", data);
// {"list": ["First", "Second"]}
}
Features
You can use the feature flag indexset
to order the CSS scraping extraction order.
Dependencies
~11–25MB
~403K SLoC