#testing #group #nextest #grouping #cargo-test #filter #test-runner

macro test-group

Organize tests into groups with macros for filtering in nextest

2 stable releases

1.0.1 Jun 17, 2023

#327 in Procedural macros

Download history 126/week @ 2023-12-17 52/week @ 2023-12-24 42/week @ 2023-12-31 117/week @ 2024-01-07 158/week @ 2024-01-14 480/week @ 2024-01-21 344/week @ 2024-01-28 413/week @ 2024-02-04 323/week @ 2024-02-11 161/week @ 2024-02-18 447/week @ 2024-02-25 340/week @ 2024-03-03 309/week @ 2024-03-10 281/week @ 2024-03-17 415/week @ 2024-03-24 258/week @ 2024-03-31

1,276 downloads per month

MIT/Apache

7KB

test-group

Organize Rust tests into groups for filtering in nextest

Crates.io CI Dependencies

Cargo's built-in test runner does not support grouping tests, which can be a problem when some of your tests are reasource-intensive and you don't want to run them all at once. Nextest solves this problem by allowing you to define test groups using filter expressions in a config file and limit concurrently.

This works, but you may find yourself:

  • Having very complex filter expressions that often go out-of-sync with code
  • Or having a special module hierarchy for your tests, separating tests into modules like heavy, dockerized etc. - this is not ideal because tests for the same component may now end up in different files, based on how "heavy" they are.

This tiny crate lets you group tests in code like so:

#[test_group::group(heavy)]
#[test]
async fn test_foo() {
  // ...
}

#[test_group::group(heavy, flaky)]
#[test]
fn test_bar() {
  // ...
}

This will result in a code like this:

mod g64dc {
  use super::*;
  mod heavy {
    use super::*;
    #[test]
    async fn test_foo() {
      // ...
    }
  }
}

mod g3d6f {
  use super::*;
  mod heavy {
    use super::*;
    mod flaky {
      use super::*;
      #[test]
      fn test_bar() {
        // ...
      }
    }
  }
}

So that these tests will appear in nextest as:

  • g3d6f::heavy::test_foo
  • g64dc::heavy::flaky::test_bar

Allowing you to filter tests by group in your .config/nextest.toml as such:

[[profile.default.overrides]]
filter = 'test(::heavy::)'
max-threads = 1

[[profile.default.overrides]]
filter = 'test(::flaky::)'
retries = 3

Or when running as:

cargo nextest run -E 'test(::flaky::)'

Note: The extra modules like g3d6f are generated by the macro to avoid module name collisions, as unlike impl blocks you cannot have more than one mod X {} in the same file. The names are based on a SHA hash of the test content to be stable but unique-ish.

Dependencies

~0.7–1.2MB
~28K SLoC