#chess #pgn

pgn-reader

Fast non-allocating and streaming reader for chess games in PGN notation

13 breaking releases

0.14.0 Mar 20, 2019
0.12.0 Nov 2, 2018
0.9.0 Jul 29, 2018
0.5.0 Mar 15, 2018
0.3.0 Nov 4, 2017

#6 in Games

Download history 16/week @ 2019-01-08 4/week @ 2019-01-15 1/week @ 2019-01-22 12/week @ 2019-01-29 4/week @ 2019-02-05 1/week @ 2019-02-12 17/week @ 2019-02-19 32/week @ 2019-02-26 9/week @ 2019-03-05 1/week @ 2019-03-12 16/week @ 2019-03-19 224/week @ 2019-03-26 18/week @ 2019-04-02 8/week @ 2019-04-09 7/week @ 2019-04-16

112 downloads per month

GPL-3.0+

7MB
7.5K SLoC

C++ 7K SLoC // 0.2% comments Rust 765 SLoC // 0.3% comments Python 221 SLoC // 0.0% comments

pgn-reader

A fast non-allocating and streaming reader for chess games in PGN notation, as a Rust library.

Build Status crates.io docs.rs

Introduction

Reader parses games and calls methods of a user provided Visitor. Implementing custom visitors allows for maximum flexibility:

  • The reader itself does not allocate (besides a single fixed-size buffer). The visitor can decide if and how to represent games in memory.
  • The reader does not validate move legality. This allows implementing support for custom chess variants, or delaying move validation.
  • The visitor can signal to the reader that it does not care about a game or variation.

Example

A visitor that counts the number of syntactically valid moves in the mainline of each game.

extern crate pgn_reader;

use std::io;
use pgn_reader::{Visitor, Skip, BufferedReader, SanPlus};

struct MoveCounter {
    moves: usize,
}

impl MoveCounter {
    fn new() -> MoveCounter {
        MoveCounter { moves: 0 }
    }
}

impl Visitor for MoveCounter {
    type Result = usize;

    fn begin_game(&mut self) {
        self.moves = 0;
    }

    fn san(&mut self, _san_plus: SanPlus) {
        self.moves += 1;
    }

    fn begin_variation(&mut self) -> Skip {
        Skip(true) // stay in the mainline
    }

    fn end_game(&mut self) -> Self::Result {
        self.moves
    }
}

fn main() -> io::Result<()> {
    let pgn = b"1. e4 e5 2. Nf3 (2. f4)
                { game paused due to bad weather }
                2... Nf6 *";

    let mut reader = BufferedReader::new_cursor(&pgn[..]);

    let mut counter = MoveCounter::new();
    let moves = reader.read_game(&mut counter)?;

    assert_eq!(moves, Some(4));
    Ok(())
}

Documentation

Read the documentation

Benchmarks (v0.12.0)

Run with lichess_db_standard_rated_2018-10.pgn (24,784,600 games, 52,750 MB uncompressed) on an SSD (Samsung 850), Intel i7-6850K CPU @ 3.60 GHz:

Benchmark Time Throughput
examples/stats.rs 111.9s 471.4 MB/s
examples/validate.rs 237.1s 222.5 MB/s
examples/parallel_validate.rs 148.6s 355.0 MB/s
scoutfish make 269.2s 196.0 MB/s
grep -F "[Event " -c 39.2s 1345.7 MB/s

examples/stats.rs with compressed files:

Compression File size Time Throughput
none 52,750 MB 111.9s 471.4 MB/s
bz2 6,226 MB 1263.1s 4.9 MB/s
xz 6,989 MB 495.9s 14.1 MB/s
gz 10,627 MB 335.7s 31.7 MB/s
lz4 16,428 MB 180.0s 91.3 MB/s

License

pgn-reader is licensed under the GPL-3.0 (or any later version at your option). See the COPYING file for the full license text.

Dependencies