#async #logging #logger

async_logger

Asyncronous logger allows writing arbitrary slices to a memory buffer, which then processed by a writer in it’s own thread

12 releases

0.3.3 Nov 7, 2020
0.3.2 Oct 27, 2020
0.3.1 Apr 19, 2020
0.2.7 Apr 8, 2020
0.1.1 Mar 14, 2020

#97 in Asynchronous

Download history 30/week @ 2020-10-31 42/week @ 2020-11-07 21/week @ 2020-11-14 23/week @ 2020-11-21 12/week @ 2020-11-28 26/week @ 2020-12-05 19/week @ 2020-12-12 7/week @ 2020-12-19 2/week @ 2020-12-26 17/week @ 2021-01-02 7/week @ 2021-01-09 13/week @ 2021-01-16 6/week @ 2021-01-23 6/week @ 2021-01-30 46/week @ 2021-02-06 64/week @ 2021-02-13

122 downloads per month
Used in async_logger_log

MIT license

65KB
1.5K SLoC

async_logger

Rust

Asynchronous logger allows fast writing of arbitrary data to a fixed size memory buffer. The data from the memory buffer then can be processed in a separate thread. This crate was created initially for logging purposes but also can be used as a queue anywhere else. See crate documentation for more info.

Implementation of log facade based on this crate is available as separate crate async_logger_log.

Intro

AsyncLoggerNB is implementation of asynchronous logger/queue that allows writing arbitrary slices to a memory buffer, and then send the buffer to a processing thread.

AsyncLoggerNB uses pair of fixed size buffers; while one buffer is being written by the multiple threads, the second is being proccessed by the single "writer" thread. Writing to a buffers is lock-free operation. Blocking appears only at the moment when buffers change their roles. This makes AsyncLoggerNB realy fast, and at the same time allows it be bounded. It can be effectively used in mutlithreaded mutlicore environment with high level of concurrent writes when you don't want to drop messages or run out of memory but still want to keep lock-free writes.

AsyncLoggerNB can process serialized data (stream of bytes) or custom complex data structures, and also references to objects.

AsyncLoggerNB can accept any "writer" as soon as it implements Writer trait. This package includes FileWriter that writes data to a file.

Examples

use async_logger::FileWriter;
use async_logger::AsyncLoggerNB;
use std::{thread, sync::Arc};

let writer = FileWriter::new("/tmp", 10*1024*1024).expect("Failed to create file writer");

let logger = Arc::new(AsyncLoggerNB::new(Box::new(writer), 8192)
    .expect("Failed to create new async logger"));

let write_line = "Hello, world!\n";

let logger_c = logger.clone();

let handle = thread::spawn(move || {

    logger_c.write_slice(write_line.as_bytes()).unwrap();
    logger_c.write_slice(write_line.as_bytes()).unwrap();
    logger_c.flush();

    logger_c.write_slice(write_line.as_bytes()).unwrap();

});

handle.join().expect("Failed on thread join");

match Arc::try_unwrap(logger) {
    Ok(logger) => logger.terminate(),
    Err(_) => panic!("Failed to terminate logger because it is still in use"),
};

Notes

  1. This crate was tested on Linux x86_64. Rust version 1.42.

No runtime deps

Features

  • metrics