6 releases
0.2.3 | Sep 10, 2024 |
---|---|
0.2.2 | Jan 24, 2024 |
0.1.2 | Jul 25, 2020 |
#362 in Algorithms
149 downloads per month
61KB
1.5K
SLoC
Ratel
Rust Implementation of a Muti-armed Bandit Simulator
The simulator consists of two components, a Bandit consisting of multiple arms each of which dispenses rewards according to a probability distribution and an Agent who pulls the arms in an attempt to learn which arm has the highest average reward.
The Bandit
A multi-armed bandit consists of a set of arms, each of which, when pulled, gives a reward according to some probability distribution. Any number of arms is allowed. There are currently five sets of distributions available; Binomial, Gaussian, Exponential, Gamma, and LogNormal.. Within those confines, all choices of distribution parameters are valid.
The Agent
The agent must determine, by some procedure, which bandit arm produces the highest average reward. There are currently three strategies implemented. The greedy algorithm always chooses the arm with the highest estimated average reward. The epsilon-greedy algorithm follows the greed algorithm most of the time, but chooses a random arm with some small probability. The optimistic algorithm chooses the arm whose estimate has the highest upper bound in some confidence range.
The Game
The Game module manages interactions between the Bandit and the Agent. The Agent pulls the Bandit's arms a certain number of times. The Game module records the wins and the rewards for each iteration.
Building Ratel
To build the simulator simply run
cargo build --release
To run the simulator, write a main
module with the desired simulation code.
Then run
cargo run --release -- ${PARAMETERS}
The simulator is designed for maximum flexibility. For inspiration, or to see how I constructed experiments, see Ratel-Experiments.
Compatibility
This code is compatible with all versions of Rust from 1.32 to 1.75.
Dependencies
~2MB
~36K SLoC