#generative-ai #actix-middleware #user-agent #crawler #ip-address #openai #block

actix_block_ai_crawling

Actix Middleware that blocks Generative AI crawlers

9 releases

0.2.8 Nov 10, 2023
0.2.7 Nov 9, 2023
0.2.3 Oct 31, 2023
0.1.0 Oct 30, 2023

#231 in HTTP client

Download history 52/week @ 2023-12-28 156/week @ 2024-01-04 45/week @ 2024-01-11 59/week @ 2024-01-18 51/week @ 2024-01-25 12/week @ 2024-02-01 22/week @ 2024-02-08 30/week @ 2024-02-15 62/week @ 2024-02-22 24/week @ 2024-02-29 20/week @ 2024-03-07 39/week @ 2024-03-14 90/week @ 2024-03-21 111/week @ 2024-03-28 31/week @ 2024-04-04 15/week @ 2024-04-11

256 downloads per month

AGPL-3.0

8KB
151 lines

Block AI crawling in Actix

Crates link: https://crates.io/crates/actix_block_ai_crawling Docs link: https://docs.rs/actix_block_ai_crawling/latest/actix_block_ai_crawling/

This blocks any HTTP requests coming from a Generative AI crawler. It works by blocking matching User Agents including ChatGPT from OpenAI, Bard from Google, and the CC crawler dataset. It also blocks OpenAI's IP addresses.

It's extremely simple to use. Just add .wrap(actix_block_ai_crawling::BlockAi); to your app.

let app = App::new()
.wrap(actix_block_ai_crawling::BlockAi);

Pull requests are welcome! Please hand-write your code. AI written code is not welcome.


lib.rs:

This crate blocks generative AI from accessing your services

It is a middleware which blocks user agents from Bard, GPT-3, and other generative AI from accessing your services. It also blocks OpenAI's crawler IP addresses.

It's extremely simple to use. Just add .wrap(actix_block_ai_crawling::BlockAi); to your app.

let app = actix_web::App()
.wrap(actix_block_ai_crawling::BlockAi);

Dependencies

~17–31MB
~541K SLoC