31 releases
0.9.0 | Aug 30, 2024 |
---|---|
0.8.2 | Jul 18, 2024 |
0.8.1 | Mar 20, 2024 |
0.7.5 | Nov 3, 2023 |
0.3.2 | Jul 31, 2019 |
#58 in Development tools
8,957 downloads per month
79KB
1.5K
SLoC
lang_tester
This crate provides a simple language testing framework designed to help when
you are testing things like compilers and virtual machines. It allows users to
express simple tests for process success/failure and for stderr/stdout, including
embedding those tests directly in the source file. It is loosely based on the
compiletest_rs
crate, but is much
simpler (and hence sometimes less powerful), and designed to be used for
testing non-Rust languages too.
For example, a Rust language tester, loosely in the spirit of
compiletest_rs
, looks as follows:
use std::{env, fs::read_to_string, path::PathBuf, process::Command};
use lang_tester::LangTester;
use tempfile::TempDir;
fn main() {
// We use rustc to compile files into a binary: we store those binary files
// into `tempdir`. This may not be necessary for other languages.
let tempdir = TempDir::new().unwrap();
LangTester::new()
.test_dir("examples/rust_lang_tester/lang_tests")
// Only use files named `*.rs` as test files.
.test_path_filter(|p| p.extension().and_then(|x| x.to_str()) == Some("rs"))
// Treat lines beginning with "#" inside a test as comments.
.comment_prefix("#")
// Extract the first sequence of commented line(s) as the tests.
.test_extract(|p| {
read_to_string(p)
.unwrap()
.lines()
// Skip non-commented lines at the start of the file.
.skip_while(|l| !l.starts_with("//"))
// Extract consecutive commented lines.
.take_while(|l| l.starts_with("//"))
.map(|l| &l[COMMENT_PREFIX.len()..])
.collect::<Vec<_>>()
.join("\n")
})
// We have two test commands:
// * `Compiler`: runs rustc.
// * `Run-time`: if rustc does not error, and the `Compiler` tests
// succeed, then the output binary is run.
.test_cmds(move |p| {
// Test command 1: Compile `x.rs` into `tempdir/x`.
let mut exe = PathBuf::new();
exe.push(&tempdir);
exe.push(p.file_stem().unwrap());
let mut compiler = Command::new("rustc");
compiler.args(&["-o", exe.to_str().unwrap(), p.to_str().unwrap()]);
// Test command 2: run `tempdir/x`.
let runtime = Command::new(exe);
vec![("Compiler", compiler), ("Run-time", runtime)]
})
.run();
}
This defines a lang tester that uses all *.rs
files in a given directory as
test files, running two test commands against them: Compiler
(i.e. rustc
);
and Run-time
(the compiled binary).
Users can then write test files such as the following:
// Compiler:
// stderr:
// warning: unused variable: `x`
// ...unused_var.rs:12:9
// ...
//
// Run-time:
// stdout: Hello world
fn main() {
let x = 0;
println!("Hello world");
}
The above file contains 4 meaningful tests, two specified by the user and
two implied by defaults: the Compiler
should succeed (e.g. return a 0
exit
code when run on Unix), and its stderr
output should warn about an unused
variable on line 12; and the resulting binary should succeed produce Hello world
on stdout
.
Integration with Cargo.
Tests created with lang_tester can be used as part of an existing test suite and
can be run with the cargo test
command. For example, if the Rust source file
that runs your lang tests is lang_tests/run.rs
then add the following to your
Cargo.toml:
[[test]]
name = "lang_tests"
path = "lang_tests/run.rs"
harness = false
Dependencies
~4–12MB
~117K SLoC