15 releases (6 breaking)

0.8.1 May 21, 2024
0.8.0 May 20, 2024
0.7.0 May 16, 2024
0.6.2 May 12, 2024
0.2.3 Apr 25, 2024

#41 in Profiling

Download history 497/week @ 2024-04-22 33/week @ 2024-04-29 692/week @ 2024-05-06 295/week @ 2024-05-13 504/week @ 2024-05-20 64/week @ 2024-05-27 30/week @ 2024-06-03 45/week @ 2024-06-10

654 downloads per month
Used in serde_json_borrow

MIT license

70KB
1.5K SLoC

Rust Docs Crates.io

Binggan

binggan logo

Binggan (餅乾, bǐng gān, means cookie in Chinese) is a benchmarking library for Rust. It is designed to be simple to use and to provide a good overview of the performance of your code and its memory consumption.

It allows arbitrary named inputs to be passed to the benchmarks.

Features

  • 📊 Peak Memory Usage
  • 💎 Stack Offset Randomization
  • 💖 Perf Integration (Linux)
  • 🔄 Delta Comparison
  • ⚡ Fast Execution
  • 🧩 Interleaving Test Runs (More accurate results)
  • 🏷️ Named Benchmark Inputs
  • 🧙 No Macros, No Magic (Just a regular API)
  • 🎨 NOW with colored output!
  • 🦀 Runs on Stable Rust

Example

use binggan::{black_box, InputGroup, PeakMemAlloc, INSTRUMENTED_SYSTEM};

#[global_allocator]
pub static GLOBAL: &PeakMemAlloc<std::alloc::System> = &INSTRUMENTED_SYSTEM;


fn test_vec(data: &Vec<usize>) {
    // ...
}
fn test_hashmap(data: &Vec<usize>) {
    // ...
}

fn bench_group(mut runner: InputGroup<Vec<usize>>) {
    runner.set_alloc(GLOBAL); // Set the peak mem allocator. This will enable peak memory reporting.

    // Enables the perf integration. Only on Linux, noop on other OS.
    runner.config().enable_perf();
    // Trashes the CPU cache between runs
    runner.config().set_cache_trasher(true);
    // Enables throughput reporting
    runner.throughput(|input| input.len() * std::mem::size_of::<usize>());
    runner.register("vec", |data| {
        black_box(test_vec(data));
    });
    runner.register("hashmap", move |data| {
        black_box(test_hashmap(data));
    });
    runner.run();
}

fn main() {
    // Tuples of name and data for the inputs
    let data = vec![
        (
            "max id 100; 100 ids all the same",
            std::iter::repeat(100).take(100).collect(),
        ),
        ("max id 100; 100 ids all different", (0..100).collect()),
    ];
    bench_group(InputGroup::new_with_inputs(data));
}

Example Output

cargo bench

turbo_buckets_vs_fxhashmap_zipfs1%
100k max id / 100k num elem
TurboBuckets                 Memory: 786.4 KB      Avg: 0.3411ms  (-8.90%)     Median: 0.3394ms  (-9.51%)     0.3223ms    0.3741ms    
Vec                          Memory: 400.0 KB      Avg: 0.0503ms  (-10.27%)    Median: 0.0492ms  (-12.27%)    0.0463ms    0.0676ms    
FxHashMap                    Memory: 442.4 KB      Avg: 1.0560ms  (+26.89%)    Median: 1.1512ms  (+58.61%)    0.6558ms    1.1979ms    
FxHashMap Reserved Max Id    Memory: 1.2 MB        Avg: 0.5220ms  (-7.86%)     Median: 0.4988ms  (-11.40%)    0.4762ms    0.7515ms    
500k max id / 500k num elem
TurboBuckets                 Memory: 4.5 MB      Avg: 1.7766ms  (+24.15%)    Median: 1.6490ms  (+15.67%)    1.3477ms    2.7288ms     
Vec                          Memory: 2.0 MB      Avg: 0.3759ms  (0.75%)      Median: 0.3598ms  (0.50%)      0.2975ms    0.5415ms     
FxHashMap                    Memory: 1.8 MB      Avg: 3.7157ms  (+6.57%)     Median: 3.5566ms  (+2.38%)     3.1622ms    5.2814ms     
FxHashMap Reserved Max Id    Memory: 9.4 MB      Avg: 5.8076ms  (+39.56%)    Median: 5.3666ms  (+31.39%)    3.0705ms    15.8945ms    

Peak Memory

To activate peak memory reporting, you need to wrap your allocator with the PeakMemAlloc and call set_alloc in the group.

While number of allocations are also interesting for performance analysis, peak memory will determine the memory requirements of the code.

TODO

  • Customize Reporting (e.g. write your own reporter)

Maybe Later Features:

  • Charts
  • Auto comparison of Histograms (e.g. if a benchmark has several bands in which it operates, it would be nice to compare them)

Dependencies

~1–9MB
~71K SLoC