3 releases
new 0.2.18 | May 1, 2025 |
---|---|
0.2.17 | May 1, 2025 |
0.2.16 | Apr 29, 2025 |
#29 in Biology
205 downloads per month
63KB
1.5K
SLoC
xsra
A performant and storage-efficient CLI tool to extract sequences from an SRA archive with support for FASTA, FASTQ, and BINSEQ outputs.
Overview
The NCBI Sequence Read Archive (SRA) is a repository of raw sequencing data.
The file format used by the SRA is a complicated binary database format that isn't directly readable by most bioinformatics tools.
This tool makes use of the ncbi_vdb
c-library through ncbi-vdb-sys
to interact with the SRA archive with safe abstractions.
This tool is designed to be a fast, storage-efficient, and more convenient replacement for the fastq-dump
and fasterq-dump
tools provided by the NCBI.
However, it is not a complete feature-for-feature replacement, and some functionality may be missing.
Features
- Multi-threaded extraction to FASTA, FASTQ, and BINSEQ records.
- Optional built-in compression of output files (FASTA, FASTQ) - [gzip, bgzip, zstd]
- Choice of BINSEQ output format (
*.bq
and*.vbq
) - Minimum read length filtering
- Technical / biological read segment selection
- Spot subsetting
- Stream directly from NCBI without intermediate prefetch
- Prefetch SRA records for faster IO
Limitations
- May not support every possible SRA archive layout (let us know if you encounter one that fails)
- Does not support all the options provided by
fastq-dump
orfasterq-dump
- Will not output sequence identifiers in the same format as
fastq-dump
orfasterq-dump
- Spot ordering is not guaranteed to be the same as the SRA archive
- Read segments are in order to keep paired-end reads together, but the order of spots is dependent on the order of completion of the threads.
- Installation bundles
ncbi-vdb
source code and builds it as a static library- This may not work on all systems
- The resulting builds will likely be system-specific and the resulting binary may not be portable.
Installation
You will need to install the rust package manager cargo
first.
# install using cargo
cargo install xsra
# validate installation
xsra --help
Usage
xsra
can either be run with on-disk accessions or can be streamed from SRA directly.
# Write all records to stdout (defaults to fastq)
xsra dump <ACCESSION>.sra
# Write all records to stdout (as fasta)
xsra dump <ACCESSION>.sra -fa
# Write all records to stdout (as fastq)
xsra dump <ACCESSION>.sra -fq
# Split records into multiple files (will create an output directory and write files there)
xsra dump <ACCESSION>.sra -s
# Split records into multiple files and compress them (gzip)
xsra dump <ACCESSION>.sra -s -cg
# Split records into multiple files, compress them (zstd), and filter out reads shorter than 11bp
xsra dump <ACCESSION>.sra -s -cz -L 11
# Write all records to stdout but only use 4 threads and compress the output (bgzip)
xsra dump <ACCESSION>.sra -T4 -cb
# Write only the first 100 spots to stdout
xsra dump <ACCESSION>.sra -l 100
# Write only segments 1 and 2 to stdout
xsra dump <ACCESSION>.sra -I 1,2
# Describe the SRA file (spot statistics)
xsra describe <ACCESSION>.sra
# Download an accession to disk
xsra prefetch <ACCESSION>.sra
You can also write BINSEQ and VBINSEQ files directly from SRA without an intermediate FASTA or FASTQ file. These operations can be done with multiple threads for faster processing as well (following same arguments as above).
# Write a BINSEQ file to (output.bq) selecting segments 1 and 2 (zero-indexed) as primary and extended.
xsra recode <ACCESSION>.sra -fb -I 0,1
# Write a BINSEQ file to (output.bq) selecting segment 3 (zero-indexed) as primary.
xsra recode <ACCESSION>.sra -fb -I 2
# Write a VBINSEQ file to (output.vbq) selecting segments 3 and 1 (zero-indexed) as primary and extended.
xsra recode <ACCESSION>.sra -fv -I 3,1
You can also use alternative data providers such as GCP
.
You will need to provide a project ID.
xsra prefetch <ACCESSION> -P gcp -G <GCP_PROJECT_ID>
Contributing
Please feel free to open an issue or pull request if you have any suggestions or improvements.
Dependencies
~52MB
~847K SLoC