|1.1.0||Sep 12, 2023|
|1.0.0||Sep 11, 2023|
#463 in Database interfaces
Rust Redis queue
rsrq is a minimal Redis-backed queue system written in Rust and inspired by RQ.
- Queues are populated by a text file containing commands to be executed (each on a new line).
- Workers can be spawned in parallel to process the queue with various modes.
- Native integration with Snakemake cluster profiles.
Feedback and contributions are welcome!
Requires Rust 1.70+
# Crates.io cargo install rsrq # or... # Bioconda conda install -c conda-forge -c bioconda rsrq
If the command
rsrq cannot be found after installation, follow these details.
🖥 CLI usage
This is not an exhaustive list of commands, help can be viewed by running the
rsrq --help command.
# Export the Redis connection string export REDIS_URL=redis://:your-password@your-endpoint-url # Enqueue commands in "/tmp/cmds.txt" to the "test" queue. rsrq enqueue test /tmp/cmds.txt # Spawn 10 workers to process the "test" queue. rsrq worker test --workers 10 # Check the status rsrq status # Purge all information from the redis database rsrq purge all
When using Snakemake integration, a cluster profile will need to be created to map the commands for
cancel. You are responsible for starting workers that will process the queue(s).
# Export the Snakemake profile rsrq snakemake config /path/to/directory # Export the Redis connection string export REDIS_URL=redis://:your-password@your-endpoint-url # Run snakemake with the cluster profile snakemake --profile /path/to/directory
You can specify a queue for each rule using the
resources.queue attribute. If none is provided
default will be used, e.g.:
rule foo: resources: queue='something'
Note: Workers do not check the CPU/memory usage of the server, it's entirely up to the user to decide how many workers can run concurrently.