1 unstable release
0.2.4 | Dec 22, 2023 |
---|
#19 in #sitemap
Used in sws-lua
75KB
2K
SLoC
Sitemap Web Scraper
Sitemap Web Scraper (sws) is a tool for simple, flexible, and yet performant web pages scraping.
It consists of a CLI written in Rust that crawls web pages and executes a Lua JIT script to scrap them, outputting results to a CSV file.
sws crawl --script examples/fandom_mmh7.lua -o result.csv
Check out the doc for more details.
lib.rs
:
HTML parsing and querying with CSS selectors.
Partial port of scraper using sws-tree which uses Rc
instead of references with lifetimes.
Dependencies
~2.8–8.5MB
~72K SLoC