#criterion #benchmark

app cargo-criterion

Cargo extension for running Criterion.rs benchmarks and reporting the results

4 releases

✓ Uses Rust 2018 edition

new 1.0.0-alpha3 Jul 6, 2020
1.0.0-alpha1 Jun 29, 2020
0.0.1 Apr 27, 2020

#35 in Cargo plugins

23 downloads per month

Apache-2.0/MIT

330KB
9K SLoC

cargo-criterion

Criterion-rs Cargo Extension

cargo-criterion is a Plugin for Cargo which handles much of the heavy lifting for analyzing and reporting on Criterion-rs benchmarks.

Table of Contents

Features

  • Statistics: Statistical analysis detects if, and by how much, performance has changed since the last benchmark run
  • Charts: Uses gnuplot or plotters to generate detailed graphs of benchmark results
  • Stable-compatible: Benchmark your code without installing nightly Rust

Quickstart

This assumes that you already have benchmarks which use Criterion-rs. If not, see the Criterion-rs Quickstart Guide. Note that Criterion.rs version 0.3.3 or later is recommended. Benchmarks which do not use Criterion.rs, or which use an earlier version, will run correctly but will not benefit from some of cargo-criterion's features.

First install cargo-criterion:

cargo install cargo-criterion --version 1.0.0-alpha3

Then you can use it to run your Criterion-rs benchmarks:

cargo criterion

Goals

  • cargo-criterion seeks to improve iteration time for Criterion-rs benchmarks. By moving functionality into a separate executable which can be installed once and reused, Criterion-rs can shrink - meaning less code to compile and link into the benchmarks themselves.
  • Because cargo-criterion can oversee the whole benchmark process from beginning to end, it's better placed to deliver features that would be difficult to implement in Criterion-rs. These include:
    • Machine-readable output
    • Tracking benchmarked performance over time

Contributing

First, thank you for contributing.

One great way to contribute to cargo-criterion is to use it for your own benchmarking needs and report your experiences, file and comment on issues, etc.

Code or documentation improvements in the form of pull requests are also welcome. If you're not sure what to work on, try checking the Beginner label.

If your issues or pull requests have no response after a few days, feel free to ping me (@bheisler).

For more details, see the CONTRIBUTING.md file.

Compatibility Policy

cargo-criterion supports the last three stable minor releases of Rust. At time of writing, this means Rust 1.41 or later. Older versions may work, but are not tested or guaranteed.

Currently, the oldest version of Rust believed to work is 1.40. Future versions of cargo-criterion may break support for such old versions, and this will not be considered a breaking change. If you require cargo-criterion to work on old versions of Rust, you will need to stick to a specific patch version of cargo-criterion.

Maintenance

cargo-criterion was originally developed and is currently maintained by Brook Heisler (@bheisler).

License

cargo-criterion is dual licensed under the Apache 2.0 license and the MIT license.

Dependencies