#web #directory #scanner #fuzzing #bruteforce

app rwalk

A blazing fast web directory scanner

35 releases

new 0.4.4 Feb 20, 2024
0.4.3 Feb 20, 2024
0.3.20 Feb 16, 2024
0.2.18 Feb 5, 2024
0.2.11 Nov 24, 2023

#135 in Testing

Download history 60/week @ 2023-11-13 253/week @ 2023-11-20 27/week @ 2023-11-27 33/week @ 2023-12-04 11/week @ 2023-12-11 68/week @ 2023-12-18 31/week @ 2023-12-25 18/week @ 2024-01-01 2/week @ 2024-01-08 36/week @ 2024-01-15 9/week @ 2024-01-22 93/week @ 2024-01-29 113/week @ 2024-02-05 1036/week @ 2024-02-12

1,251 downloads per month

MIT license

94KB
2K SLoC

rwalk

Crates.io GitHub Release

A blazing fast web directory scanner written in Rust. It's like dirsearch but faster and with less features. It is designed to be fast in recursive scans and to be able to handle large wordlists.

Unlike other tools, rwalk does not provide advanced fuzzing features such as parameter fuzzing, header discovery, etc.

Features

  • Multi-threaded
  • Recursive directory scanning
  • Save progress to resume later
  • Cherry-pick responses (filter by status code, length, etc.)
  • Custom wordlists (merge multiple wordlists, filter out words, etc.)
  • Write results to file (JSON, CSV, etc.)
  • Configurable request parameters (headers, cookies, etc.)
  • Request throttling
  • Proxy support

Installation

From homebrew

brew install cestef/tap/rwalk

From crates.io

cargo install rwalk

From source

git clone https://github.com/cestef/rwalk.git
cd rwalk
cargo install --path .

You can also download the latest binary from the releases page.

Development

With just

just run https://example.com wordlist.txt

With cargo

cargo run --release -- https://example.com wordlist.txt

Usage

You can run rwalk --help or read the help file for more information.

Response Filtering

To cherry-pick the responses, you can use the --filter (-f) flags to filter specific responses. For example, to only show responses that contain admin:

rwalk ... --filter contains:admin

or only requests that took more than 1 second:

rwalk ... --filter "time:>1000"

Available filters:

  • starts: <STRING>
  • ends: <STRING>
  • contains: <STRING>
  • regex: <REGEX>
  • length: <RANGE>
  • status: <RANGE>
  • time: <RANGE>
  • hash: <STRING> (MD5)

Each filter can be negated by adding a ! before the filter. For example, to exclude responses that contain admin:

rwalk ... --filter "!contains:admin"

You can also filter only at specific depths with the [depth]filter format. For example, to only show responses that contain admin at depth 2:

rwalk ... --filter "[2]contains:admin"

[!NOTE] Depth starts at 0.

Inputting ranges

In some cases , you may want to input a <RANGE> of values. You can use the following formats:

Format Description
5 Exactly 5
5-10 Between 5 and 10 (inclusive)
5,10 Exactly 5 or 10
>5 Greater than 5
<5 Less than 5
5,10,15 Exactly 5, 10, or 15
>5,10,15 Greater than 5, or exactly 10 or 15
5-10,15-20 Between 5 and 10 or between 15 and 20 (inclusive)

Wordlists

You can pass multiple wordlists to rwalk. For example:

rwalk https://example.com wordlist1.txt wordlist2.txt

rwalk will merge the wordlists and remove duplicates. You can also apply filters and transformations to the wordlists (see below).

You can also pass wordlists from stdin:

cat wordlist.txt | rwalk https://example.com -

[!NOTE] A checksum is computed for the wordlists and stored in case you abort the scan. If you resume the scan, rwalk will only load the wordlists if the checksums match. See Saving and Resuming scans for more information.

Wordlist Filters

You can filter words from the wordlist by using the --wordlist-filter (-w) flag. For example, to only use words that start with admin:

rwalk ... --wordlist-filter starts:admin

Available filters:

  • starts: <STRING>
  • ends: <STRING>
  • contains: <STRING>
  • regex: <REGEX>
  • length: <RANGE>

Wordlist Transformations

To quickly modify the wordlist, you can use the --transform flag. For example, to add a suffix to all words in the wordlist:

rwalk ... --transform suffix:.php

To replace all occurrences of admin with administrator:

rwalk ... --transform replace:admin=administrator

Available transformations:

  • prefix: <STRING>
  • suffix: <SUFFIX>
  • remove: <STRING>
  • replace: <OLD=NEW>
  • upper
  • lower
  • capitalize
  • reverse

Additional response details

If you need more details about the matched responses, you can use the --show flag. For example, to show the body hash and length:

rwalk ... --show hash --show length 

Available details:

  • length
  • hash
  • headers
  • body
  • headers_length
  • headers_hash

Scanning modes

Recursive scan

By default rwalk will use a recursive-like scan. You can change the depth of the scan with the --depth (-d) flag:

rwalk https://example.com wordlist.txt -d 3

Classic scan

A more traditional scan can be done with the --mode classic flag:

rwalk https://example.com/$ wordlist.txt --mode classic

Notice that the $ character is used to indicate the position of the wordlist in the URL. This character can be changed with the --fuzz-key flag.

Permutations

In case you want to explore more complex URL structures, you can use the classic mode in combination with the --permutations flag, which will generate all possible permutations of the wordlist:

rwalk https://example.com/$/abcd/$ wordlist.txt --mode classic --permutations

This will generate all possible combinations of the wordlist in the URL, e.g.:

https://example.com/word1/abcd/word1 https://example.com/word1/abcd/word2 https://example.com/word1/abcd/word3 https://example.com/word2/abcd/word1 https://example.com/word2/abcd/word2

...

Interactive mode

You can use the --interactive (-i) flag to enter interactive mode. In this mode, you can set parameters one by one and run the scan when you're ready.

Available commands:

  • set <PARAM> <VALUE>: Set a parameter
  • append <PARAM> <VALUE>: Append a value to a Vec parameter
  • unset <PARAM>: Unset a parameter
  • list: Show the current parameters
  • run: Run the scan
  • exit: Exit interactive mode
  • help: Show help
  • clear: Clear the screen

Output

By default, rwalk will print the results to the terminal. You can also save the results to a file with the --output (-o) flag:

rwalk https://example.com wordlist.txt -o results.json

Available output formats:

  • *.json
  • *.csv
  • *.md
  • *.txt

Throttling

The throttling value will be multiplied by the number of threads. For example, if you have 10 threads and a throttling value of 5, the total number of requests per second will be 50.

rwalk https://example.com wordlist.txt --throttle 5 -t 10 

Saving and resuming scans

By default, if you abort the scan with Ctrl + C, rwalk will save the progress to a file called .rwalk.json. You can resume the scan by running with --resume:

rwalk --resume

If you want to save the progress to a different file, you can use the --save-file flag:

rwalk https://example.com wordlist.txt --save-file myscan.json 

The auto-saving behavior can be disabled with --no-save.

Proxy support

You can pass a proxy URL with the --proxy flag:

rwalk https://example.com wordlist.txt --proxy http://pro.xy:8080

Authentication is also supported with --proxy-auth:

rwalk https://example.com wordlist.txt --proxy http://pro.xy:8080 --proxy-auth username:password

Passing parameters as environment variables

You can pass parameters as environment variables. For example, to set the number of threads to 10:

THREADS=10 rwalk https://example.com wordlist.txt

is equivalent to:

rwalk https://example.com wordlist.txt -t 10

The env file located at ~/.config/rwalk/.env will be loaded automatically.

Examples

Basic scan

rwalk https://example.com wordlist.txt

Recursive scan

rwalk https://example.com wordlist.txt -d 3

Warning: Recursive scans can take a long time and generate a lot of traffic. Use with caution.

Custom headers/cookies

rwalk https://example.com wordlist.txt -H "X-Forwarded-For: 203.0.113.195" -c "session=1234567890"

Follow redirects

rwalk https://example.com wordlist.txt -R 2

Custom request body

rwalk https://example.com wordlist.txt -X POST -D '{"username": "admin", "password": "admin"}'

FAQ

Where can I find wordlists?

How do I get support?

Open an issue or ask in the Discord server.

Is rwalk stable?

rwalk is stable but it's still in the early stages of development. It should work for most use cases but there may be bugs.

Benchmarks

The following benchmarks were run on a 2023 MacBook Pro with an M3 Pro chip on a 10 Gbps connection via WiFi. The target was http://ffuf.me/cd/basic and the wordlist was common.txt.

Each tool was run 10 times with 100 threads. The results are below:

Command Mean [s] Min [s] Max [s] Relative
rwalk 2.406 ± 0.094 2.273 2.539 1.00
dirsearch 8.528 ± 0.149 8.278 8.743 3.54 ± 0.15
ffuf 2.552 ± 0.181 2.380 3.005 1.06 ± 0.09

If you want to run the benchmarks yourself, you can use the bench command:

just bench

Positional arguments can also be passed.

just bench <URL> <WORDLIST_FILE> <THREADS>

Contributing

If you want to contribute to rwalk, please read the CONTRIBUTING.md file.

License

Licensed under the MIT License.

Dependencies

~18–35MB
~568K SLoC