engram

Engram is a fast and space-efficient version control system for portable file backups, inspired by git and rsnapshot. It creates snapshots of directories and stores them in a compressed and portable delta-based format.

Engram can be used in a cron job to automatically backup files, and the backup can be stored remotely with a tool like rclone. It will only update internal files when necessary, so modification times can be factored when using rsync-like tools. Engram does not encrypt snapshots, so this should be handled externally if desired.

Unlike other backup tools, engram allows for deletion of any number of previous revisions, as it stores instructions on how to create previous snapshots from the current state.

Usage

ℹ️ See engram --help for more detailed information.

Initialize a repository with data:
$ engram init ~/repo ~/data

Update a repository with new data:
$ engram update ~/repo ~/data 3days

Pull historical repository data:
$ engram pull ~/repo ~/out 2024-11-28T12:00:00

Revert updates to a repository:
$ engram revert ~/repo 2024-11-28T12:00:00

Manually remove old updates:
$ engram clean ~/repo 3days

File Structure

repo/
├── .ignore        <-- [optional ignorelist]
├── .lock          <-- [in-progress lockfile]
├── base/
│   ├── file1.txt  <-- [current files]
│   └── file2.txt 
├── hashes         <-- [hash table]
└── patches/
    ├── 1736634596/
    │   ├── 0      <-- [objects]
    │   ├── 2
    │   └── index  <-- [patch table]
    └── 1736634679/
        ├── 1
        └── index  

Build Instructions

Cargo (stable):
$ RUSTFLAGS="-C target-cpu=native" cargo install engram

Nix (latest):
$ git clone https://gitlab.com/mbednarek360/engram
$ cd engram; nix build

Current Limitations

• renamed directories can create big patch tables
• cannot pull files bigger than available system memory
• will not preserve file metadata
• only supports a UNIX-based OS