Test In Docker (tid)

Table of Contents

1. Introduction
2. Dependencies
3. Installation
4. Usage
   1. Grouping tests
5. Contributing

Introduction

While developping another project, I wanted to add integration tests to it, but I wanted all of the tests to be run in a clean environment to avoid test making each other fail/succeed.
I also needed to test them on different linux distributions to ensure that they would work on all of them.
Because of that I decided to use Docker as it would allow me to create multiple environments easily.

Once I had decided to use Docker, I looked for a way to run integration tests using the containers, but I couldn't find any that would fit my needs (most of them were just running all the tests in a single container).
So I decided to create my own test runner that would allow me to run the tests in multiple containers.

Dependencies

This project requires the following dependencies:

• Any linux distribution (you can try it on other OS but it might not work)
• Docker engine
• A project to test

Installation

Compile from source

To compile the project from source, you will need to have the following dependencies:

• git
• Rust

The rust toolchain has multiple programs, in this case we will need cargo to compile the project.

First, clone the repository:

git clone https://git@gitlab.com:Rignchen/test-in-docker.git
cd test-in-docker

Then, compile the project:

cargo build --release

Once the project is compiled, you can find the binary in the target/release directory.
To make it easier to run the program, you can create a symbolic link to the binary in your ~/.local/bin directory:

ln -s $(pwd)/target/release/tid ~/.local/bin/tid

Or you can move the binary to a directory that is in your PATH:

mv target/release/tid ~/.local/bin/tid

This will allow you to run the program from anywhere by just typing tid in your terminal.

Install using cargo

To install the project using cargo, you will need to have the following dependencies:

• Rust

The rust toolchain has multiple programs, in this case we will need cargo to install the project.

To install the project using cargo, you can use the following command:

cargo install test-in-docker

Usage

This program uses docker to run the tests, you can define the tests in a Dockerfile, the default file searched by the program is Dockerfile.test but you can give it a different name if you want.

Here is an example of a Dockerfile that uses this program:

FROM archlinux:latest AS base

FROM base AS success-test
# DockerTest: success
RUN sh -c 'exit 0'

FROM base AS fail-test
# DockerTest: fail
RUN sh -c 'exit 1'

FROM base AS success-test-error
# DockerTest: success
RUN sh -c 'exit 1'

FROM base AS fail-test-error
# DockerTest: fail
RUN sh -c 'exit 0'

As you can see, the tests have a special comment that starts with DockerTest:
This comment is used to tell the program what you expect to happen when the test is run.
The program will log a ✔ if the test ended with the expected result and a ✘ if it didn't.
The possible values for the test are:

• success: Both the build and the run should succeed (exit code 0)
• fail: Either the build or the run should fail (exit code 1-255)

To run the tests, you can use the following command:

tid
tid --dockerfile Dockerfile.test # Use a different Dockerfile

The following Dockerfile would output the following results:

base ✘
├── success-test ✔
├── fail-test ✔
├── success-test-error ✘
└── fail-test-error ✘

Grouping tests

In the example above, all the images were made from base, but you can group them by making them use different images.
This is usefull to make the output more digest.

This is also the recommended way to run the tests if your tests have different needs as docker won't lose as much time preparing the environment for each test.

Here is an example of a Dockerfile that uses this program:

FROM archlinux:latest AS base
RUN pacman -Syu --noconfirm

FROM base AS successes
RUN pacman -S --noconfirm python

FROM base AS mixte
RUN pacman -S --noconfirm python

FROM base AS fails

FROM successes AS success-test1
# DockerTest: success
RUN echo "print('Success')" | python

FROM mixte AS success-test2
# DockerTest: success
RUN echo "print('Success')" | python

FROM mixte AS fail-test
# DockerTest: fail
RUN echo "print('Fail')" | python

FROM fails AS success-test-error
# DockerTest: success
RUN echo "print('Success')" | python

This Dockerfile would output the following results:

base ✘
├── successes ✔
│   └── success-test1 ✔
├── mixte ✘
│   ├── success-test2 ✔
│   └── fail-test ✘
└── fails ✘
    └── success-test-error ✘

As you can see, the tests are grouped by the image they are made from.
Also, if all tests from an image succeed, the name of the individual tests is not displayed.

Contributing

If you want to contribute to this project, you can do so by creating a merge request on the GitLab repository.

If you do contribute to the project, please make sure to

• Format the code with cargo fmt
• Lint the code with cargo clippy
• Write tests for all new features (I will look at the tests before merging the code)
• Run the tests with cargo test -- --include-ignored to make sure that they all pass (some tests cannot be run using the CI because they require docker, that's why some tests are ignored)
• Update any documentation that needs to be updated