redis-keyspace-stats

This program analyzes your Redis keyspace and returns statistics about it. It's somewhat flexible. Usually, you'll provide glob-style patterns to group keys into bins.

This tool is under development, and some obviously useful features are not yet implemented.

Sampling modes:

• ✅  Random sampling
• 🚧  Random sampling of keys matching a pattern
• ✅  Scan all keys
• 🚧  Scan all keys matching a pattern

Statistics:

• ✅  Memory: total, total estimated, 50/90/99th percentiles
• ✅  TTL: percent with a TTL, 50/90/99th percentiles
• ✅  Data type breakdown

Output formats:

• ✅  Summary pretty-printed table to STDOUT
• 🚧  Summary CSV/TSV
• 🚧  Summary HTML
• 🚧  Raw data CSV/TSV

Redis support:

• ✅  Over TLS (rediss:// connection strings)
• ✅  Logical databases, specified in the connection string (redis://host:port/db)
• 🚧  Clusters

⚠️ Warnings

• This is NOT production-grade software (yet?). Use at your own risk.
• Redis is single-threaded, so be careful about running this against busy production systems. This tool sends commands to Redis in batches (and sleeps between them) to reduce its impact, but caution should still be exercised. Start with a small sample size, like the default -n 100 before running more comprehensive analysis.
• Use the --sample=all mode with care; the -n option is ignored and ALL keys will be enumerated with SCAN
• This tool fetches keys, and includes a handful of them in its output. While it's rare to store sensitive information in Redis keys, it's worth noting. Values are never fetched or included in any output.
• The flags/options/organization/etc of the CLI will change as more sampling modes and other features are added

Installation

For now, there are no published binaries of this tool.

1. Make sure you have Rust installed, or get it via rustup
2. Run cargo install redis-keyspace-stats to download + compile the binary

Usage examples

Invoking the CLI with -h (or --help, which is more verbose) will print documentation:

$ redis-keyspace-stats -h
Usage: redis-keyspace-stats [OPTIONS] [PATTERNS]...

Arguments:
  [PATTERNS]...  Glob-style patterns to group keys together

Options:
      --sample <SAMPLE_MODE>             [default: random] [possible values: all, random]
  -n, --samples <N_SAMPLES>              Ignored when --sample=all is specified [default: 100]
      --batch-size <BATCH_SIZE>          [default: 100]
      --batch-sleep-ms <BATCH_SLEEP_MS>  [default: 100]
      --stats <STATS>                    [default: memory,ttl] [possible values: memory, ttl, type]
  -o, --out <OUTPUT_MODE>                [default: table] [possible values: table]
      --url <URL>                        [default: redis://127.0.0.1]
  -h, --help                             Print help
  -V, --version                          Print version

Let's get some quick memory + TTL stats, sampling 50 keys:

$ redis-keyspace-stats --url $REDIS_URL -n 50
Sampled 50 of 128 keys in db0
┌─────────┬────────────────┬────────────────────┬────────────────────────┬─────────────────┐
│ Pattern │ Keys           │ Example keys       │ Memory                 │ TTL             │
├─────────┼────────────────┼────────────────────┼────────────────────────┼─────────────────┤
│ *       │ 50 counted     │ company:121#memes  │ 360.77 kB (sum)        │ 54.00% have TTL │
│         │ 128 est. total │ user:79#memes      │ 923.57 kB (est. total) │ 2m 47s (p50)    │
│         │                │ user:8#memes       │ 7.75 kB (p50)          │ 3m 57s (p90)    │
│         │                │ user:92#friends    │ 11.9 kB (p90)          │ 4m 13s (p99)    │
│         │                │ company:84#friends │ 12.87 kB (p99)         │                 │
└─────────┴────────────────┴────────────────────┴────────────────────────┴─────────────────┘

Using what's showing in the "Example keys" column, let's write a few glob-style patterns to bin keys together:

$ redis-keyspace-stats --url $REDIS_URL -n 50 'user:*#messages' 'user:?#*' 'company:*'
Sampled 50 of 128 keys in db0
┌─────────────────┬───────────────┬────────────────────┬────────────────────────┬──────────────────┐
│ Pattern         │ Keys          │ Example keys       │ Memory                 │ TTL              │
├─────────────────┼───────────────┼────────────────────┼────────────────────────┼──────────────────┤
│ user:*#messages │ 8 counted     │ user:27#messages   │ 30.27 kB (sum)         │ 37.50% have TTL  │
│                 │ 20 est. total │ user:34#messages   │ 77.5 kB (est. total)   │ 1m 30s (p50)     │
│                 │               │ user:13#messages   │ 3.4 kB (p50)           │ 3m 25s (p90)     │
│                 │               │ user:58#messages   │ 6.22 kB (p90)          │ 3m 51s (p99)     │
│                 │               │ user:60#messages   │ 6.22 kB (p99)          │                  │
├─────────────────┼───────────────┼────────────────────┼────────────────────────┼──────────────────┤
│ user:?#*        │ 2 counted     │ user:3#friends     │ 1.25 kB (sum)          │ 100.00% have TTL │
│                 │ 5 est. total  │ user:8#memes       │ 3.19 kB (est. total)   │ 1m 52s (p50)     │
│                 │               │                    │ 624 B (p50)            │ 2m 39s (p90)     │
│                 │               │                    │ 822.4 B (p90)          │ 2m 50s (p99)     │
│                 │               │                    │ 867.04 B (p99)         │                  │
├─────────────────┼───────────────┼────────────────────┼────────────────────────┼──────────────────┤
│ company:*       │ 24 counted    │ company:1#messages │ 162.13 kB (sum)        │ 45.83% have TTL  │
│                 │ 61 est. total │ company:75#memes   │ 415.05 kB (est. total) │ 2m 51s (p50)     │
│                 │               │ company:10#friends │ 7.24 kB (p50)          │ 3m 58s (p90)     │
│                 │               │ company:69#memes   │ 11.85 kB (p90)         │ 4m 2s (p99)      │
│                 │               │ company:6#memes    │ 12.64 kB (p99)         │                  │
├─────────────────┼───────────────┼────────────────────┼────────────────────────┼──────────────────┤
│ *               │ 16 counted    │ user:123#memes     │ 149.02 kB (sum)        │ 31.25% have TTL  │
│                 │ 41 est. total │ user:71#friends    │ 381.5 kB (est. total)  │ 2m 10s (p50)     │
│                 │               │ user:25#friends    │ 10.05 kB (p50)         │ 2m 59s (p90)     │
│                 │               │ user:86#memes      │ 12.86 kB (p90)         │ 3m 4s (p99)      │
│                 │               │ user:110#friends   │ 13.29 kB (p99)         │                  │
└─────────────────┴───────────────┴────────────────────┴────────────────────────┴──────────────────┘

Note that the first pattern that matches a key will determine the group.

Development

Testing locally

• Seed some fake test data via environment variable: RKS_SEED_FAKE_DATA=true cargo run -- --sample=all 'company:*'
• Starting the redis-cli binary and running monitor can be useful for debugging

Releasing

1. Bump the version in Cargo.toml
2. Run cargo test
3. Commit, add a git tag for the release, and push
4. Run cargo publish