Lib.rs

› Compression › Fluvio
#data-streaming #fluvio #data-processing #data-pipeline #zstd #consumer

fluvio-compression

Fluvio Compression library

by Sehyo Chang and 64 contributors

13 releases

0.3.4 Oct 9, 2024
0.3.3 Jul 22, 2024
0.3.2 Oct 31, 2023
0.3.0 May 26, 2023
0.1.1 Mar 25, 2022

#184 in Compression

Download history 507/week @ 2024-08-17 591/week @ 2024-08-24 759/week @ 2024-08-31 777/week @ 2024-09-07 341/week @ 2024-09-14 1186/week @ 2024-09-21 736/week @ 2024-09-28 871/week @ 2024-10-05 516/week @ 2024-10-12 396/week @ 2024-10-19 929/week @ 2024-10-26 1191/week @ 2024-11-02 544/week @ 2024-11-09 727/week @ 2024-11-16 597/week @ 2024-11-23 779/week @ 2024-11-30

2,796 downloads per month
Used in 11 crates (3 directly)

Apache-2.0

34KB
748 lines

Fluvio Compression

Library with handlers to compress and uncompress data in the fluvio protocol.

In fluvio, compression is done in producer side, then consumers and SPU when it is using SmartModules, uncompress the data using the compression information that is in the attributes of the batch.

Currently, the supported compressions codecs are None (default), Gzip, Snappy, Zstd and LZ4.

Dependencies

~2.8–4.5MB
~82K SLoC