#benchmark

benchman

A benchmark tool that focuses on one-shot benchmark

9 releases

0.2.6 Dec 19, 2021
0.2.5 Dec 18, 2021
0.1.1 Dec 16, 2021

#335 in Development tools

Download history 5/week @ 2022-12-02 13/week @ 2022-12-09 15/week @ 2022-12-16 12/week @ 2022-12-23 4/week @ 2022-12-30 6/week @ 2023-01-06 5/week @ 2023-01-13 16/week @ 2023-01-20 12/week @ 2023-01-27 23/week @ 2023-02-03 24/week @ 2023-02-10 48/week @ 2023-02-17 18/week @ 2023-02-24 9/week @ 2023-03-03 21/week @ 2023-03-10 6/week @ 2023-03-17

56 downloads per month
Used in 2 crates

MIT license

10KB
200 lines

benchman

Crates.io documentation

Features

  • Focus on one-shot benchmark
  • RAII-style
  • Statistics (Average, Median, 95% and 99% percentile)
  • Colored output
  • Tagging
  • Nesting

Motivation

I guess there are two types of benchmarks.

One is a benchmark of a small and fast function in which we want the statistics from a million of iterations. For this type of benchmark, Criterion.rs is a good fit.

Another type is what I call one-shot benchmark.

You may have wanted to write a benchmark program like this.

let mut db = DB::new();

let t = Instant::now();
db.write(...);
println!("write: {:?}", t.elapsed());

let t = Instant::now();
db.read(...);
println!("read: {:?}", t.elapsed());

According to Criterion.rs #531, this type of benchmark is infeasible with Criterion.rs because Criterion is focusing on the first type.

That's why I started to create benchman.

RAII-style measurement

RAII is a good technique to manage resource access. My idea behind designing benchman is that stopwatch is like a resource because it is like a producer of a benchmark result that sends the result to the single central consumer and there is a strict rule that stopwatch shouldn't send the result twice.

With this idea, the library is designed like this.

let stopwatch = benchman.get_stopwatch("some_tag");
do_something();
drop(stopwatch);

// or

{
    let _sw = benchman.get_stopwatch("some_tag");
    do_something();
}

When the stopwatch is dropped, the measurement result is sent to the central database.

Screenshot

スクリーンショット 2021-12-18 12 42 40

Author

Akira Hayakawa (@akiradeveloper)

Dependencies

~470–660KB
~12K SLoC