62 releases (5 breaking)
new 0.7.5 | Apr 24, 2024 |
---|---|
0.6.3 | Apr 19, 2024 |
0.6.0 | Mar 23, 2024 |
0.2.13 | Dec 22, 2023 |
0.2.11 | Nov 24, 2023 |
#280 in Web programming
575 downloads per month
210KB
4.5K
SLoC
A blazing fast web directory scanner written in Rust. It's like dirsearch but faster and with less features. It is designed to be fast in recursive scans and to be able to handle large wordlists.
Unlike other tools, rwalk does not provide advanced fuzzing features such as parameter fuzzing, header discovery, etc.
Features
- Multi-threaded
- Recursive directory scanning
- Save progress to resume later
- Cherry-pick responses (filter by status code, length, etc.)
- Advanced filters (regex, json, ranges, etc.)
- Custom wordlists (merge multiple wordlists, filter out words, etc.)
- Write results to file (JSON, CSV, etc.)
- Configurable request parameters (headers, cookies, etc.)
- Request throttling
- Proxy support
Table of Contents
Installation
From homebrew
brew install cestef/tap/rwalk
From crates.io
cargo install rwalk
From source
git clone https://github.com/cestef/rwalk.git
cd rwalk
cargo install --path .
You can also download the latest binary from the releases page.
Usage
You can run rwalk --help
or read the help file for more information.
Scanning modes
Recursive scan
By default rwalk
will use a recursive-like scan. You can change the depth of the scan with the --depth
(-d
) flag:
rwalk https://example.com wordlist.txt -d 3
The recursive mode only scans urls identified as directories. If you are not happy with the default behavior, you can use the --force-recursion
(--fr
) flag to force the recursion on all found urls.
Classic scan
In case you want to fuzz more precise paths, you can either use the --mode classic
flag or provide a fuzzing placeholder in the URL. For example:
rwalk https://example.com/$ wordlist.txt
Notice that the $
character is used to indicate the position of the wordlist in the URL.
This character is the default, but it can be changed with the following format:
rwalk https://example.com/W1 wordlist.txt:W1
In this case, the W1
string will be replaced by the first word in the wordlist.
You can also use this to replace with multiple wordlists:
rwalk https://example.com/W1/W2 wordlist1.txt:W1 wordlist2.txt:W2
W1
will be replace by the words in wordlist1.txt
and W2
by the words in wordlist2.txt
.
To use the same wordlist in multiple places, you can use the following format:
rwalk https://example.com/W1/W2 wordlist.txt:W1,W2
This will generate all possible combinations of the wordlist at W1
and W2
.
Response Filtering
To cherry-pick the responses, you can use the --filter
(-f
) flags to filter specific responses. For example, to only show responses that contain admin
:
rwalk ... --filter contains:admin
or only requests that took more than 1
second:
rwalk ... --filter "time:>1000"
Available filters:
starts
:<STRING>
ends
:<STRING>
contains
:<STRING>
regex
:<REGEX>
length
:<RANGE>
status
:<RANGE>
time
:<RANGE>
hash
:<STRING>
(MD5)
Each filter can be negated by adding a !
before the filter. For example, to exclude responses that contain admin
:
rwalk ... --filter "!contains:admin"
You can also filter only at specific depths with the [depth]filter
format. For example, to only show responses that contain admin
at depth 2
:
rwalk ... --filter "[2]contains:admin"
[!NOTE] Depth starts at
0
.
Additional response details
If you need more details about the matched responses, you can use the --show
flag. For example, to show the body hash and length:
rwalk ... --show hash --show length
Available details:
length
hash
headers
body
headers_length
headers_hash
type
Ranges
In some cases , you may want to input a <RANGE>
of values.
You can use the following formats:
Format | Description |
---|---|
5 |
Exactly 5 |
5-10 |
Between 5 and 10 (inclusive) |
5,10 |
Exactly 5 or 10 |
>5 |
Greater than 5 |
<5 |
Less than 5 |
5,10,15 |
Exactly 5 , 10 , or 15 |
>5,10,15 |
Greater than 5 , or exactly 10 or 15 |
5-10,15-20 |
Between 5 and 10 or between 15 and 20 (inclusive) |
Wordlists
You can pass multiple wordlists to rwalk
. For example:
rwalk https://example.com wordlist1.txt wordlist2.txt
rwalk
will merge the wordlists and remove duplicates. You can also apply filters and transformations to the wordlists (see below).
You can also pass wordlists from stdin:
cat wordlist.txt | rwalk https://example.com -
[!NOTE] A checksum is computed for the wordlists and stored in case you abort the scan. If you resume the scan,
rwalk
will only load the wordlists if the checksums match. See Saving and Resuming scans for more information.
Filters
You can filter words from the wordlist by using the --wordlist-filter
(-w
) flag. For example, to only use words that start with admin
:
rwalk ... --wordlist-filter starts:admin
The filters are applied to all wordlists by default. This behavior can be controlled with the [key]filter
format. For example, to only use words that start with admin
from the second wordlist:
rwalk wordlist1.txt:W1 wordlist2.txt:W2 -w "[W2]starts:admin"
Available filters:
starts
:<STRING>
ends
:<STRING>
contains
:<STRING>
regex
:<REGEX>
length
:<RANGE>
Transformations
To quickly modify the wordlist, you can use the --transform
(-T
) flag. For example, to add a suffix to all words in the wordlist:
rwalk ... --transform suffix:.php
To replace all occurrences of admin
with administrator
:
rwalk ... --transform replace:admin=administrator
Just as in the wordlist filters, the transformations are applied to all wordlists by default. This behavior can be controlled with the [key]transform
format. For example, to add a suffix to all words in the second wordlist:
rwalk wordlist1.txt:W1 wordlist2.txt:W2 -T "[W2]suffix:.php"
Available transformations:
prefix
:<STRING>
suffix
:<SUFFIX>
remove
:<STRING>
replace
:<OLD=NEW>
upper
lower
capitalize
reverse
Miscellaneous
Interactive mode
You can use the --interactive
(-i
) flag to enter interactive mode. In this mode, you can set parameters one by one and run the scan when you're ready.
Available commands:
set <PARAM> <VALUE>
: Set a parameterappend <PARAM> <VALUE>
: Append a value to a Vec parameterunset <PARAM>
: Unset a parameterlist
: Show the current parametersrun
: Run the scanexit
: Exit interactive modehelp
: Show helpclear
: Clear the screen
Output
By default, rwalk
will print the results to the terminal. You can also save the results to a file with the --output
(-o
) flag:
rwalk https://example.com wordlist.txt -o results.json
Available output formats:
*.json
*.csv
*.md
*.txt
Throttling
The throttling value will be multiplied by the number of threads. For example, if you have 10
threads and a throttling value of 5
, the total number of requests per second will be 50
.
rwalk https://example.com wordlist.txt --throttle 5 -t 10
Saving and resuming scans
By default, if you abort the scan with Ctrl + C, rwalk will save the progress to a file called .rwalk.json
. You can resume the scan by running with --resume
:
rwalk --resume
If you want to save the progress to a different file, you can use the --save-file
flag:
rwalk https://example.com wordlist.txt --save-file myscan.json
The auto-saving behavior can be disabled with --no-save
.
Proxy support
You can pass a proxy URL with the --proxy
flag:
rwalk https://example.com wordlist.txt --proxy http://pro.xy:8080
Authentication is also supported with --proxy-auth
:
rwalk https://example.com wordlist.txt --proxy http://pro.xy:8080 --proxy-auth username:password
Passing parameters from a config
The configuration file located at ~/.config/rwalk/config.toml
will be loaded by default. You can also pass a custom configuration file with the --config
flag:
rwalk https://example.com wordlist.txt --config myconfig.toml
Examples
Basic scan
rwalk https://example.com wordlist.txt
Recursive scan
rwalk https://example.com wordlist.txt -d 3
Warning: Recursive scans can take a long time and generate a lot of traffic. Use with caution.
Custom headers/cookies
rwalk https://example.com wordlist.txt -H "X-Forwarded-For: 203.0.113.195" -c "session=1234567890"
Follow redirects
rwalk https://example.com wordlist.txt -R 2
Custom request body
rwalk https://example.com wordlist.txt -X POST -D '{"username": "admin", "password": "admin"}'
FAQ
Where can I find wordlists?
How do I get support?
Open an issue or ask in the Discord server.
Is rwalk stable?
rwalk is stable but it's still in the early stages of development. It should work for most use cases but there may be bugs.
Where can I test this tool?
You can use the ffuf.me website to test rwalk. I also wrote a blog post about it with a few cool examples.
Benchmarks
The following benchmarks were run on a 2023 MacBook Pro with an M3 Pro chip on a 10 Gbps connection via WiFi. The target was http://ffuf.me/cd/basic and the wordlist was common.txt.
Each tool was run 10
times with 100
threads. The results are below:
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
rwalk |
2.406 ± 0.094 | 2.273 | 2.539 | 1.00 |
dirsearch |
8.528 ± 0.149 | 8.278 | 8.743 | 3.54 ± 0.15 |
ffuf |
2.552 ± 0.181 | 2.380 | 3.005 | 1.06 ± 0.09 |
If you want to run the benchmarks yourself, you can use the bench
command:
just bench
Positional arguments can also be passed.
just bench <URL> <WORDLIST_FILE> <THREADS>
Please take these results with a grain of salt.
"There are three types of lies: lies, damn lies and benchmarks"
Contributing
Contributions are welcome! I am always looking for new ideas and improvements.
If you want to contribute to rwalk, please read the CONTRIBUTING.md file.
Make sure that your commits follow the Conventional Commits standard. This project uses commitizen to help you with that.
License
Licensed under the MIT License.
Dependencies
~22–40MB
~642K SLoC