13 releases

0.3.2 Dec 31, 2020
0.3.1 Dec 31, 2020
0.3.0 Sep 20, 2020
0.2.2 Oct 29, 2019
0.1.2 Dec 21, 2018

#36 in Compression

Download history 172/week @ 2020-09-29 52/week @ 2020-10-06 10/week @ 2020-10-13 15/week @ 2020-10-20 2/week @ 2020-10-27 15/week @ 2020-11-03 1/week @ 2020-11-10 15/week @ 2020-11-17 12/week @ 2020-11-24 4/week @ 2020-12-01 24/week @ 2020-12-08 21/week @ 2020-12-15 67/week @ 2020-12-29 17/week @ 2021-01-05 3/week @ 2021-01-12

65 downloads per month

Custom license

23KB
417 lines

compressed_log

Latest Version Documentation

A Rust crate to compress logs on the fly, and send them over the network after reaching a configured threshold.

This is useful for when you have a fleet of embedded devices running a Rust program, instead of building a complicated metrics framework you can collect the normal Rust log output securely using https and efficiently using Deflate compression.

The pure rust backend of LibFlate2 is used with only a 100kb in memory buffer. Using the 'fast' compression setting the average compression ratio in our tests is ~25 without any noticible cpu impact on even embedded MIPS processors.

Right now compressed_log pulls in the full Actix suite to handle futures and async requests. We hope to transition it to native futures and dramaticly reudce the dependency tree once they become more mature.

On the server side logs are simply dumped into a file, where they can be aggregated and processed by standard log collection tools.

Features

  • Uses log API
  • LieFlate2 to compress log on the fly
  • Configurable threshold which will trigger in-memory log rotation and data transmission.
  • Fully architecture portable, including Mips and other BE architectures

Example

Check out examples folder for more examples. Including custom formats for collected logs.

Client:

let level = Level::Info;
let logger = LoggerBuilder::new()
    .set_level(level)
    .set_compression_level(Compression::Slow)
    .set_sink_url("http://127.0.0.1:8000/sink/")
    .build()?;
log::set_boxed_logger(Box::new(logger))?;
log::set_max_level(level.to_level_filter());

Server:

The current server reference implementaiton is compressed_log_sink

Dependencies

~23MB
~536K SLoC