#log #logging #async #channel #mpsc #crossbeam #double #efficiency

fast_log

Rust async log High-performance asynchronous logging

12 stable releases

1.5.17 Jun 20, 2022
1.5.1 Mar 7, 2022
1.4.12 Dec 30, 2021
1.3.32 Sep 26, 2021
1.0.3 Mar 8, 2020

#35 in Debugging

Download history 509/week @ 2022-03-14 587/week @ 2022-03-21 219/week @ 2022-03-28 232/week @ 2022-04-04 151/week @ 2022-04-11 189/week @ 2022-04-18 431/week @ 2022-04-25 414/week @ 2022-05-02 277/week @ 2022-05-09 1235/week @ 2022-05-16 455/week @ 2022-05-23 774/week @ 2022-05-30 305/week @ 2022-06-06 278/week @ 2022-06-13 261/week @ 2022-06-20 143/week @ 2022-06-27

1,101 downloads per month
Used in 3 crates

MIT license

59KB
1.5K SLoC

log

#中文

Build Status GitHub release

the fast log . This crate uses #! [forbid(unsafe_code)] to ensure everything is implemented in 100% Safe Rust.

A log implementation for extreme speed, using Crossbeam to double the efficiency (as opposed to the standard library MPSC) with a lockless channel, using a channel to write logs asynchronously. Completely use the safe code to achieve, without safe code

  • Low overhead, based on thread

  • High performance, use lockless message queue, log is stored in queue, then flush disk. It does not block the caller

  • Full APPEND mode file writing, high efficiency for solid state/mechanical disk (solid state and mechanical disk sequential write performance is better than random write)

  • When channel pressure increases, logs can be written in batches at a time

  • Built-in ZIP compression, compressed file name date + serial number, no need to worry about the log file is too large

  • Built-in log segmentation, custom log full number of immediately split

  • Built-in filtering configuration support, can be customized to filter out other library printed logs

  • Support custom compression algorithms, such as ZIP and LZ4


              -----------------
log data->    | main channel(crossbeam)  |   ->          
              ----------------- 
                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender1  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender2  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender3  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender4  |
                                        ----------------                                    ----------------------


How fast is?

//MACOS(Apple M1MAX-32GB)
Time: 221.079ms ,each:221 ns/op
QPS: 4523139 Iter/s

support Future mode,async await based on mpsc channel, tokio or async_std support log split,zip_compress

  • how to use?
log = "0.4"
fast_log = {version = "1.5"}

or enable zip/lz4/gzip Compression library

log = "0.4"
# "lz4","zip","gzip"
fast_log = {version = "1.5" , features = ["lz4","zip","gzip"]}

Use Log(Console)

use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().console()).unwrap();
    log::info!("Commencing yak shaving{}", 0);
}

Use Log(File)

use fast_log::{init_log};
use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().file("target/test.log")).unwrap();
    log::info!("Commencing yak shaving{}", 0);
    info!("Commencing yak shaving");
}

Split Log(.log packer)

use fast_log::plugin::file_split::RollingType;
use fast_log::consts::LogSize;
use fast_log::plugin::packer::LogPacker;

#[test]
pub fn test_file_compation() {
    fast_log::init(Config::new()
        .console()
        .file_split("target/logs/",
                    LogSize::MB(1),
                    RollingType::All,
                    LogPacker{})).unwrap();
    for _ in 0..200000 {
        info!("Commencing yak shaving");
    }
    log::logger().flush();
}
Custom Log(impl do_log method)
use fast_log::{LogAppender};
use log::{error, info, warn};

pub struct CustomLog{}
impl LogAppender for CustomLog{
    fn do_log(&mut self, record: &FastLogRecord) {
        print!("{}",record);
    }
}
fn  main(){
    let wait = fast_log::init(Config::new().custom(CustomLog {})).unwrap();
    info!("Commencing yak shaving");
    log::logger().flush();
}

Dependencies

~1.7–3MB
~60K SLoC