#file-extension #decompression #gzip #zstd #bzip2 #tokio #file-format

autocompress

Automatically select suitable decoder from magic bytes or encoder from file extension

8 releases (4 breaking)

0.6.0 Dec 24, 2023
0.5.1 Dec 4, 2023
0.4.2 Nov 16, 2023
0.2.2 Jul 18, 2021
0.1.0 Apr 1, 2020

#76 in Compression

Download history 15/week @ 2023-12-18 6/week @ 2023-12-25 4/week @ 2024-01-08 10/week @ 2024-01-22 13/week @ 2024-02-05 16/week @ 2024-02-12 100/week @ 2024-02-19 33/week @ 2024-02-26 26/week @ 2024-03-04 40/week @ 2024-03-11 26/week @ 2024-03-18 79/week @ 2024-03-25 82/week @ 2024-04-01

233 downloads per month
Used in proxy-rs

Apache-2.0

210KB
4.5K SLoC

autocompress-rs

Build GitHub GitHub top language Crates.io Docs.rs

Automatically select suitable decoder from magic bytes or encoder from file extension. This library also provides I/O thread pool to perform decompression and compression in background threads.

Supported file formats

Feature flags

  • gzip : Gzip format support
  • bgzip : bgzip format support
  • bzip2 : Bzip2 format support
  • xz : XZ format support
  • zstd : Zstd format support
  • rayon : Off-load compression and decompression process to another thread using rayon
  • tokio : Async reader and writer support with tokio
  • tokio_fs: Enable autodetect_async_open function

Example

Read from a file

use std::io::prelude::*;
use autocompress::autodetect_open;

fn main() -> anyhow::Result<()> {
    let mut reader = autodetect_open("testfiles/pg2701.txt.xz")?;
    let mut buf = Vec::new();
    reader.read_to_end(&mut buf)?;
    Ok(())
}

Write to a file

use std::io::prelude::*;
use autocompress::{autodetect_create, CompressionLevel};

fn main() -> anyhow::Result<()> {
    let mut writer = autodetect_create("target/doc-index.xz", CompressionLevel::Default)?;
    writer.write_all(&b"Hello, world\n"[..])?;
    Ok(())
}

Compress file in parallel

use std::io::prelude::*;
use autocompress::{autodetect_parallel_create, CompressionLevel};

fn main() -> anyhow::Result<()> {
  let mut writer = autodetect_parallel_create("target/doc-index2.xz", CompressionLevel::Default)?;
  writer.write_all(&b"Hello, world\n"[..])?;
  Ok(())
}

Dependencies

~0.4–4MB
~72K SLoC