#url #irc #harvest #search #generate-html #irssi

bin+lib urlharvest

URL harvester for IRC irssi logs, no bot

6 releases

0.5.5 Nov 3, 2021
0.5.4 Oct 21, 2021
0.5.2 Sep 29, 2021

#4 in #harvest

MIT/Apache

50KB
1.5K SLoC

Rust 1K SLoC // 0.0% comments Tera 355 SLoC Handlebars 75 SLoC Shell 9 SLoC // 0.2% comments

URL harvester for IRC, no bot

  • Harvest URLs from irssi IRC client logs, insert into SQLite db
  • Fetch metadata, e.g. title and update db.
  • Generate html pages.
  • Implement a search page

Please note: this harvester is tailing your IRC client (irssi) logs on disk and does not need or include an ircbot of any kind. Consider running your irssi on a cloud vm to stay "always connected" :-)

Any other kind of chat log source would be trivial to implement. Basically, the chat text is just scanned with regex match and detected URLs are saved & indexed.

Dependencies

~60MB
~1M SLoC