Border

A reinforcement learning library in Rust.

Border consists of the following crates:

• border-core provides basic traits and functions generic to environments and reinforcmenet learning (RL) agents.
• border-tensorboard has TensorboardRecorder struct to write records which can be shown in Tensorboard. It is based on tensorboard-rs.
• border-py-gym-env is a wrapper of the Gymnasium environments written in Python.
• border-atari-env is a wrapper of atari-env, which is a part of gym-rs.
• border-tch-agent is a collection of RL agents based on tch, including Deep Q network (DQN), implicit quantile network (IQN), and soft actor critic (SAC).
• border-async-trainer defines some traits and functions for asynchronous training of RL agents by multiple actors, which runs sampling processes in parallel. In each sampling process, an agent interacts with an environment to collect samples to be sent to a shared replay buffer.

You can use a part of these crates for your purposes, though border-core is mandatory. This crate is just a collection of examples. See Documentation for more details.

News

The owner of this repository will be changed from taku-y to laboroai.

Status

Border is experimental and currently under development. API is unstable.

Examples

There are some example sctipts in border/examples directory. These are tested in Docker containers, speficically the one in aarch64 directory on M2 Macbook air. Some scripts take few days for the training process, tested on Ubuntu22.04 virtual machine in GPUSOROBAN, a computing cloud.

Docker

In docker directory, there are scripts for running a Docker container, in which you can try the examples described above. Currently, only aarch64 is mainly used for the development.

Tests

The following command has been tested in the Docker container running on M2 Macbook air.

cargo test --features=tch

License

lib.rs:

Derive macros for making newtypes of types that implements border_core::Obs, border_core::Act and order_core::replay_buffer::SubBatch.

These macros will implements some conversion traits for combining interfaces of an environment and an agent.