robotstxt is used at run time in 4 crates.

Depender (with downloads and own rev deps) robotstxt version
150 voyager ^0.3
crawly ^0.3
sqlite-robotstxt ^0.3.0
wls ^0.3.0