robotstxt is used at run time in 4 crates.
| Depender | robotstxt version | |||
|---|---|---|---|---|
| voyager | ^0.3 | |||
| wls | ^0.3.0 | |||
| crawly | ^0.3 | |||
| sqlite-robotstxt | ^0.3.0 | |||
robotstxt is used at run time in 4 crates.
| Depender | robotstxt version | |||
|---|---|---|---|---|
| voyager | ^0.3 | |||
| wls | ^0.3.0 | |||
| crawly | ^0.3 | |||
| sqlite-robotstxt | ^0.3.0 | |||