#log #performance #fast #logging #mpsc #async #channel #crossbeam #double

fast_log

Rust async log High-performance asynchronous logging

48 stable releases

new 1.4.15 Jan 14, 2022
1.4.12 Dec 30, 2021
1.3.32 Sep 26, 2021
1.3.24 Jul 15, 2021
1.0.3 Mar 8, 2020

#27 in Debugging

Download history 181/week @ 2021-09-27 95/week @ 2021-10-04 163/week @ 2021-10-11 97/week @ 2021-10-18 85/week @ 2021-10-25 206/week @ 2021-11-01 129/week @ 2021-11-08 127/week @ 2021-11-15 111/week @ 2021-11-22 156/week @ 2021-11-29 309/week @ 2021-12-06 214/week @ 2021-12-13 128/week @ 2021-12-20 199/week @ 2021-12-27 145/week @ 2022-01-03 181/week @ 2022-01-10

674 downloads per month
Used in 3 crates

MIT license

37KB
1K SLoC

log

#中文

Build Status GitHub release

the fast log . This crate uses #! [forbid(unsafe_code)] to ensure everything is implemented in 100% Safe Rust.

A log implementation for extreme speed, using Crossbeam to double the efficiency (as opposed to the standard library MPSC) with a lockless channel, using a channel to write logs asynchronously. Completely use the safe code to achieve, without safe code

  • Low overhead, based on May coroutines

  • High performance, use lockless message queue, log is stored in queue, then flush disk. It does not block the caller

  • Full APPEND mode file writing, high efficiency for solid state/mechanical disk (solid state and mechanical disk sequential write performance is better than random write)

  • Built-in ZIP compression, compressed file name date + serial number, no need to worry about the log file is too large

  • Built-in log segmentation, custom log full number of immediately split

  • Built-in filtering configuration support, can be customized to filter out other library printed logs

  • Support custom compression algorithms, such as ZIP and LZ4


              -----------------
log data->    | main channel(crossbeam)  |   ->          
              ----------------- 
                                        ----------------                                    ----------------------
                                  ->    |coroutines channel(may)|  -> background coroutines  |    appender1  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |coroutines channel(may)|  -> background coroutines  |    appender2  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |coroutines channel(may)|  -> background coroutines  |    appender3  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |coroutines channel(may)|  -> background coroutines  |    appender4  |
                                        ----------------                                    ----------------------


How fast is >?

// Win10 (PC 6 core, mechanical hard disk)

  • Use QPS: 525892 /s

// Win10 (PC 6 core, SSD)

  • USE QPS: 508215 pieces /s

support Future mode,async await based on mpsc channel, tokio or async_std support log split,zip_compress

  • how to use?
log = "0.4"
#default is enable zip packer
fast_log = {version = "1.4"}

or

log = "0.4"
#default is enable zip packer,this is allow lz4 packer(this is vary faster)
fast_log = {version = "1.4" , features = ["lz4"]}

Use Log

use fast_log::{init_log};
use log::{error, info, warn};
fn  main(){
    fast_log::init_log("requests.log",  log::Level::Info, None,true);
    info!("Commencing yak shaving");
}
split log, allow_zip_compress = Zip compression
#[test]
pub fn test_file_compation() {
    init_split_log("target/logs/",  LogSize::MB(1), false, log::Level::Info, None, Box::new(ZipPacker{}), true);// or Box::new(LZ4Packer{})
    for _ in 0..200000 {
        info!("Commencing yak shaving");
    }
    sleep(Duration::from_secs(1));
}
Custom Log
use fast_log::{init_custom_log,LogAppender};
use log::{error, info, warn};

pub struct CustomLog{}
impl LogAppender for CustomLog{
    fn do_log(&mut self, record: &FastLogRecord) {
        print!("{}",record);
    }
}
fn  main(){
    fast_log::init_custom_log(vec![Box::new(CustomLog {})],  log::Level::Info, Box::new(NoFilter {}));
    info!("Commencing yak shaving");
    sleep(std::time::Duration::from_secs(1));
}

Dependencies

~2.2–4MB
~80K SLoC