robotstxt is used at run time in 4 crates.
Depender (with downloads and own rev deps) | robotstxt version | |||
---|---|---|---|---|
160 | voyager | ^0.3 | ||
110 | sqlite-robotstxt | ^0.3.0 | ||
crawly | ^0.3 | |||
wls | ^0.3.0 |
robotstxt is used at run time in 4 crates.
Depender (with downloads and own rev deps) | robotstxt version | |||
---|---|---|---|---|
160 | voyager | ^0.3 | ||
110 | sqlite-robotstxt | ^0.3.0 | ||
crawly | ^0.3 | |||
wls | ^0.3.0 |