Stockfish Binpack

Rust port of the Stockfish binpack reader from the C++ version.

Binpacks store chess positions and their evaluations in a compact format. Instead of storing complete positions, they store the differences between moves. This makes them very space efficient - using only 2.5 bytes per position on average. See Anatomy for more details.

Compile

If your machine has the fast BMI2 instruction set (Zen 3+), you should enable the feature flag

cargo build --release --features bmi2;

or define it in your Cargo.toml file (change version).

[dependencies]
binpack = { version = "0.4.3", features = ["bmi2"] }

Usage

Run the following Cargo command in your project directory:

cargo add sfbinpack

use sfbinpack::CompressedTrainingDataEntryReader;

fn main() {
    let file = File::open("data.binpack").unwrap();
    let mut reader = CompressedTrainingDataEntryReader::new(file).unwrap();

    while reader.has_next() {
        let entry = reader.next();

        println!("entry:");
        println!("fen {}", entry.pos.fen().unwrap());
        println!("uci {:?}", entry.mv.as_uci());
        println!("score {}", entry.score);
        println!("ply {}", entry.ply);
        println!("result {}", entry.result);
        println!("\n");
    }
}

More examples can be found in the examples directory.
If you are doing some counting keep in mind to use a u64 type for the counter.

Examples

To run the examples in the examples directory, use the following command:

cargo run --release --example <example_name>

binpack_reader - Read a binpack file and print the contents. binpack_writer - Write a binpack file from a list of positions.

Performance Comparison

Slightly faster when compiled with bmi2 because of _pdep_u64 trick which is missing in the upstream version.

Anatomy

Compression

When compressing new data, it is advised to store the entire continuation of the actual game. This will allow for a much better compression ratio.
Failure to do so will result in a larger file size, than compared to other alternatives.

License

GNU General Public License v3.0

https://www.gnu.org/licenses/gpl-3.0.html