3 releases

0.1.2 Dec 18, 2020
0.1.1 Nov 30, 2020
0.1.0 Nov 30, 2020

#511 in HTTP client

MIT license

26KB

Voight Kampff Crates.io

This crate is a rust version of this excellent ruby gem for figuring out if a http request is from a bot/crawler/scraper/replicant.

It relies only on the user agent that was sent and does no other checks on its own.

Usage

voight_kampff::bot() will return true if the user agent matches a user agent in the crawler-user-agents.json file

In your Cargo.toml:

voight_kampff = "0.1.2"

In your code:

use voight_kampff;

voight_kampff::bot("Mozilla/5.0 ...");

As with the ruby version, this crate gets its list of user agents from this repo by monperrus.

Dependencies

~2.5–4MB
~74K SLoC