|0.1.6||Nov 5, 2020|
|0.1.5||Nov 3, 2020|
|0.1.2||Oct 29, 2020|
#108 in Testing
44 downloads per month
This is a tool for parallelizing a VK-GL-CTS or dEQP run across the CPUs in a system, collecting the results, and comparing them to the baseline for the device. It is geared toward driver developers and CI systems looking for fast turnaround of dEQP results (it is not a valid way to produce a VK/GL conformance result).
apt-get install cargo cargo install deqp-runner
deqp-runner run \ --deqp ~/src/VK-GL-CTS-build/modules/gles2/deqp-gles2 \ --caselist ~/src/VK-GL-CTS/external/openglcts/data/mustpass/gles/aosp_mustpass/3.2.6.x/gles2-master.txt \ --output new-run \ --baseline last-run/failures.csv \ -- \ --deqp-surface-width=256 --deqp-surface-height=256 \ --deqp-surface-type=pbuffer \ --deqp-gl-config-name=rgba8888d24s8ms0 \ --deqp-visibility=hidden
You will get summary output as tests are run and results collected, with failing tests logged to stderr with references of where to find the associated caselists (and the full QPA file next to them). The failing tests will also be extracted to testname.qpa which you can run dEQP's testlog-to-xml on for visualizing the failure.
Unexpectedly failing tests will be automatically rerun to detect flaky tests, which will be treated as success of the pipeline (this is useful in CI, to keep rare and uncategorized flakes from impacting other developers). Additionally, you can pass a --flakes argument to a list of regexes of known flaky tests, to reduce the chance of spurious failures in a CI pipeline.
Add the following to ~/.cargo/config:
[target.armv7-unknown-linux-gnueabihf] linker = "arm-linux-gnueabihf-gcc" [target.aarch64-unknown-linux-gnu] linker = "aarch64-linux-gnu-gcc"
And set up the new toolchain and build:
rustup target add aarch64-unknown-linux-gnu cargo build --release --target aarch64-unknown-linux-gnu deqp-runner scp target/aarch64-unknown-linux-gnu/release/deqp-runner device:bin/