#favicon #logo #website #scraper #cli

bin+lib site_icons

Website icon scraper that fetches sizes (with WASM support)

12 releases

0.1.11 May 6, 2022
0.1.10 May 3, 2022
0.1.8 Apr 2, 2022
0.1.6 Feb 3, 2021
0.1.1 Jan 30, 2021

#205 in Images

Download history 3/week @ 2022-01-25 3/week @ 2022-02-01 3/week @ 2022-02-08 4/week @ 2022-02-15 12/week @ 2022-02-22 2/week @ 2022-03-01 5/week @ 2022-03-08 23/week @ 2022-03-15 22/week @ 2022-03-22 45/week @ 2022-03-29 12/week @ 2022-04-05 5/week @ 2022-04-12 5/week @ 2022-04-19 82/week @ 2022-04-26 162/week @ 2022-05-03 188/week @ 2022-05-10

437 downloads per month
Used in 2 crates

GPL-3.0 license

30KB
837 lines

site_icons

Crates.io Documentation GitHub Sponsors

An efficient website icon scraper for rust or command line usage.

Features

  • Super fast!
  • Partially downloads images to find the sizes
  • Can extract a site logo <img> using a weighing system
  • Works with inline-data URIs (and automatically converts <svg> to them)
  • Supports WASM (and cloudflare workers)

Command line usage

cargo install site_icons

site-icons https://github.com
# https://github.githubassets.com/favicons/favicon.svg site_favicon svg
# https://github.githubassets.com/app-icon-512.png app_icon png 512x512
# https://github.githubassets.com/app-icon-192.png app_icon png 192x192
# https://github.githubassets.com/apple-touch-icon-180x180.png app_icon png 180x180

Rust usage

use site_icons::Icons;

let mut icons = Icons::new();
// scrape the icons from a url
icons.load_website("https://github.com").await?;

// fetch all icons, ensuring they exist & determining size
let entries = icons.entries().await;

// entries are sorted from highest to lowest resolution
for icon in entries {
  println!("{:?}", icon)
}

Sources

  • HTML favicon tag (or looking for default /favicon.ico)
  • Web app manifest icons field
  • <img> tags on the page, directly inside the header OR with a src|alt|class containing the text "logo"

Running locally

Install cargo make and then:

cargo make run https://github.com

Dependencies

~11–21MB
~423K SLoC