1 stable release

1.0.0 Dec 22, 2023

#29 in #pages


Used in 2 crates (via sws-scraper)

MIT/Apache

26KB
680 lines

Sitemap Web Scraper

Sitemap Web Scraper (sws) is a tool for simple, flexible, and yet performant web pages scraping.

It consists of a CLI written in Rust that crawls web pages and executes a Lua JIT script to scrap them, outputting results to a CSV file.

sws crawl --script examples/fandom_mmh7.lua -o result.csv

Check out the doc for more details.


lib.rs:

SlotMap-backed ID-tree.

Port of ego-tree, but using Rc instead of references with lifetimes, and without using unsafe.

Dependencies

~280KB