github-discussions-mirror

A tool to:

1. continuously mirror a GitHub discussions page into a folder of JSON
2. download all images referenced in discussions, optionally
3. generate a static site from that folder, optionally

For a demo, see https://github.com/untitaker/rayhunter-discussions-mirror

Installation

cargo build --release
# or
cargo install github-discussions-mirror

Usage

Syncing data

export GITHUB_TOKEN=your_token_here

./target/release/github-discussions-mirror sync -r owner/repo

Your data is in ./data. JSON and images.

You can periodically rerun this command and it will sync new changes from GitHub to ./data efficiently.

Generating a static site

Based on the files in ./data, you can now generate a static site:

./target/release/github-discussions-mirror generate

The HTML files are now in ./site.

Static site generation is entirely based on ./data's contents, i.e. fully offline. It generates the entire site from scratch each time.

To have a live mirror, you have to retain the ./data folder, sync it periodically, then generate the static site from it each time.

Re-doing sync

For various reasons you may want to re-do the data sync. Mostly when bugs are fixed in this tool.

You can pass --force to sync to sync entirely from scratch.

You can also do --since-days=4 to reset the checkpoint by 4 days and refetch only discussions that have seen activity since 4 days before the last sync.

License

MIT (see Cargo.toml and LICENSE)