#cli #stress-testing #testing #cp-tool #testing-tool

app quicktest

Command Line Interface (CLI) for stress testing for competitive programming contest

26 releases (9 breaking)

new 1.0.0-rc.5 May 19, 2022
0.10.0 Apr 28, 2022
0.7.0 Jan 7, 2022
0.6.0 Dec 16, 2021
0.5.1 Jun 22, 2021

#4 in Command line utilities

Download history 4/week @ 2022-01-29 20/week @ 2022-02-19 1/week @ 2022-03-05 35/week @ 2022-03-12 69/week @ 2022-03-19 28/week @ 2022-04-09 58/week @ 2022-04-16 82/week @ 2022-04-23 115/week @ 2022-04-30 170/week @ 2022-05-07 485/week @ 2022-05-14

863 downloads per month

MIT license

140KB
3.5K SLoC

Quick Test CLI

Command Line Interface (CLI) for Stress Testing for Competitive Programming

Current Crates.io Version Quick Test License: MIT

Demo

cmp gif

Documentation

Table of Contents

Getting Started

Installation

Installation tutorial for Linux, Windows and macOS can be found here on the docs website

Introduction

Quick Test CLI is a project to perform stress testing in competitive programming contests in an easy and fast way, focusing only on the contest.

Currently, Quick Test CLI supports three types of tests which are listed below:

quicktest cmp quicktest stress quicktest check
cmp gif stress gif check gif
  • quicktest cmp | qt cmp: It checks the correctness of the algorithm we want to verify by comparing it with a brute force solution which is usually very slow, but is 100% sure to provide the correct solution.

    • Sample:

      quicktest cmp --target-file=main.cpp --correct-file=correct.cpp --gen-file=gen.cpp
      # Or shorter:
      qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tout 1000 --tc 1000
      
  • quicktest stress | qt stress: Verify that the code execution time does not exceed what is allowed, using a random generator for multiple test cases.

    Note: In this scenario there is no slower solution with which to compare the correction.

    • Sample:
      quicktest stress --target-file=main.cpp --gen-file=gen.cpp
      # Or shorter:
      qt stress -t main.cpp -g gen.cpp --tout 1000 --tc 1000
      
  • quicktest check | qt check: In some problems more than one answer is accepted, so the quicktest cmp command would not work correctly, in this case a script checker is used to verify the correctness of the algorithm.

    • Sample:
      quicktest check --target-file=main.cpp --checker-file=correct.cpp --gen-file=gen.cpp
      # Or shorter:
      qt check -t main.cpp -c check.cpp -g gen.cpp --tout 1000 --tc 1000
      
  • quicktest output | qt output: run all test cases that match a prefix and save the response to an output file

    • Sample:
      quicktest output --target-file=main.cpp --prefix=testcase_ac
      # Or shorter:
      qt output -t main.cpp -p test_cases/testcase_ac --tout 1000
      

Commands

Note: you can use the long command quicktest or the alias qt

  • quicktest cmp | qt cmp

    Required Options

    • -t=<value> | --target-file=<value>
    • -c=<value> | --correct-file=<value>
    • -g=<value> | --gen-file=<value>

    Other Options

    • --test-cases=<value> | --tc=<value> [default: 1000]
    • --timeout=<value> | --tout=<value> [default: 2000] Unit of time: ms
    • --memory-limit=<value> | --ml=<value> [default: 1000000000 - 1GB] Unit of time: bytes
    • --prefix=<value> | -p=<value> conflict with --gen-file (Only one can be used at a time)
    • --diff Show differences between the expected file and the output file

  • quicktest stress | qt stress

    Required Options

    • -t=<value> | --target-file=<value>
    • -g=<value> | --gen-file=<value>

    Other Options

    • --test-cases=<value> | --tc=<value> [default: 1000]
    • --timeout=<value> | --tout=<value> [default: 2000] Unit of time: ms
    • --memory-limit=<value> | --ml=<value> [default: 1000000000 - 1GB] Unit of time: bytes
    • --prefix=<value> | -p=<value> conflict with --gen-file (Only one can be used at a time)

  • quicktest check | qt check

    Required Options

    • -t=<value> | --target-file=<value>
    • -c=<value> | --checker-file=<value>
    • -g=<value> | --gen-file=<value>

    Other Options

    • --test-cases=<value> | --tc=<value> [default: 1000]
    • --timeout=<value> | --tout=<value> [default: 2000] Unit of time: ms
    • --memory-limit=<value> | --ml=<value> [default: 1000000000 - 1GB] Unit of time: bytes
    • --prefix=<value> | -p=<value> conflict with --gen-file (Only one can be used at a time)

  • Flags of the cmp, stress and check subcommands

    • --break-bad | --break Break if WA, TLE or RTE states occurs
    • --run-ac Run test cases Accepted
    • --run-all Run all test cases
    • --run-rte Run test cases Run Time Error
    • --run-tle Run test cases Time Limited Exceeded
    • --run-wa Run test cases Wrong Answer
    • --save-all Save all test cases
    • --save-bad Save only bad cases with WA, TLE or RTE states

  • quicktest output | qt output

    Required Options

    • -t=<value> | --target-file=<value>
    • -p=<value> | --prefix=<value>

    Other Options

    • --timeout=<value> | --tout=<value> [default: 2000] Unit of time: ms
    • --memory-limit=<value> | --ml=<value> [default: 1000000000 - 1GB] Unit of time: bytes
    • --break-bad | --break Break if WA, TLE or RTE states occurs
    • --save-out Save the output of the target file for each test case

  • quicktest setup | qt setup

    Subcommand

    • config Subcommand that allows to change C++ settings

      Options

      • --label=<value> Label with the path of the configuration that you want to change
      • --value=<value> value you want to change a selected label to

  • quicktest example | qt example

    Flags

    • --check Show examples of the check subcommand
    • --cmp Show examples of the cmp subcommand
    • --stress Show examples of the stress subcommand
    • --output Show examples of the output subcommand
    • --setup Show examples of the setup subcommand

    Nota: can only use one flag at a time

How to Contribute

If you are interested in contributing to the Quick Test CLI project, please take a look at the Contribute guide

Bug reports

You can report any bugs here.

Supported Languages

Language
C++
Java
Python
Rust Lang
Go Lang
GNU C
Kotlin

Compilation and Execution Commands

Language Compile / Interpreter Execution Command
C++17 g++ -std=c++17 -Wall -DONLINE_JUDGE=1 -o main main.cpp ./main
Java javac -d .qt/ Main.java java -cp .qt/ Main
Python3 python3 main.py
Rust Lang cp main.rs ~/.quicktest/rust/src/main.rs && cargo build --release --quiet --manifest-path ~/.quicktest/rust/Cargo.toml && cp ~/.quicktest/rust/target/release/rust main ./main
Go Lang cp main.go ~/.quicktest/go_mod/main.go && go build -buildmode=exe -o ./.qt/main ~/.quicktest/go_mod/main.go ./main
GNU C gcc -std=gnu11 main.c -o .qt/main ./main
Kotlin kotlinc main.kt -include-runtime -d .qt/main.jar java -jar .qt/main.jar

License

Licensed under either of these:


Logo image based on the one made by Freepik for www.flaticon.com

Documentation was based on cpbooster

Dependencies

~5–12MB
~224K SLoC