robotxt is used at run time in 1 crate.

Depender robotxt version
muffy ^0.6.1