Brunch

Brunch is a very simple Rust micro-benchmark runner inspired by easybench. It has roughly a million times fewer dependencies than criterion, does not require nightly, and maintains a (single) "last run" state for each benchmark, allowing it to show relative changes from run-to-run.

(The formatting is also quite pretty.)

As with all Rust benchmarking, there are a lot of caveats, and results might be artificially fast or slow. For best results:

• Build optimized;
• Collect lots of samples;
• Repeat identical runs to get a feel for the natural variation;

Brunch cannot measure time below the level of a nanosecond, so if you're trying to benchmark methods that are really fast, you may need to wrap them in a method that runs through several iterations at once. For example:

use brunch::Bench;

///# Generate Strings to Test.
fn string_seeds() -> Vec<String> {
    (0..10_000_usize).into_iter()
        .map(|i| "x".repeat(i))
        .collect()
}

///# Generate Strings to Test.
fn byte_seeds() -> Vec<Vec<u8>> {
    (0..10_000_usize).into_iter()
        .map(|i| "x".repeat(i).into_bytes())
        .collect()
}

brunch::benches!(
    Bench::new("String::len(_)")
        .run_seeded_with(string_seeds, |vals| {
            let mut len: usize = 0;
            for v in vals {
                len += v.len();
            }
            len
        }),
    Bench::new("Vec::len(_)")
        .run_seeded_with(byte_seeds, |vals| {
            let mut len: usize = 0;
            for v in vals {
                len += v.len();
            }
            len
        }),
);

Installation

Add brunch to your dev-dependencies in Cargo.toml, like:

[dev-dependencies]
brunch = "0.8.*"

Benchmarks should also be defined in Cargo.toml. Just be sure to set harness = false for each:

[[bench]]
name = "encode"
harness = false

The following optional environmental variables are supported:

Usage

The heart of Brunch is the Bench struct, which defines a single benchmark. There isn't much configuration required, but each Bench has the following:

The struct uses builder-style methods to allow everything to be set in a single chain. You always need to start with Bench::new and end with one of the runner methods — Bench::run, Bench::run_seeded, or Bench::run_seeded_with. If you want to change the sample or timeout limits, you can add Bench::with_samples or Bench::with_timeout in between.

There is also a special Bench::spacer method that can be used to inject a linebreak into the results. See below for an example.

Examples

The benches! macro is the easiest way to run Brunch benchmarks.

Simply pass a comma-separated list of all the Bench objects you want to run, and it will handle the setup, running, tabulation, and give you a nice summary at the end.

By default, this macro will generate the main() entrypoint too, but you can suppress this by adding "inline:" as the first argument.

Anyhoo, the default usage would look something like the following:

use brunch::{Bench, benches};

// Example benchmark adding 2+2.
fn callback() -> Option<usize> { 2_usize.checked_add(2) }

// Example benchmark multiplying 2x2.
fn callback2() -> Option<usize> { 2_usize.checked_mul(2) }

// Let the macro handle everything for you.
benches!(
    Bench::new("usize::checked_add(2)")
        .run(callback),

    Bench::new("usize::checked_mul(2)")
        .run(callback2),
);

When declaring your own main entrypoint, you need to add "inline:" as the first argument. The list of Bench instances follow as usual after that.

use brunch::{Bench, benches};

/// # Custom Main.
fn main() {
    // A typical use case for the "inline" variant would be to declare
    // an owned variable for a benchmark that needs to return a reference
    // (to e.g. keep Rust from complaining about lifetimes).
    let v = vec![0_u8, 1, 2, 3, 4, 5];

    // The macro call goes here!
    benches!(
        inline:

        Bench::new("vec::as_slice()").run(|| v.as_slice()),
    );

    // You can also do other stuff afterwards if you want.
    eprintln!("Done!");
}

For even more control over the flow, skip the macro and just use Benches directly.

Interpreting Results

If you run the example benchmark for this crate, you should see a summary like the following:

Method                         Mean    Change        Samples
------------------------------------------------------------
fibonacci_recursive(30)     2.22 ms    +1.02%    2,408/2,500
fibonacci_loop(30)         56.17 ns       ---    2,499/2,500

The Method column speaks for itself, but the numbers deserve a little explanation: