46 releases (14 breaking)

Uses new Rust 2021

0.15.0 Nov 23, 2022
0.13.0 Aug 4, 2022
0.12.0 May 23, 2022
0.10.0 Mar 31, 2022
0.2.0 Nov 28, 2019

#84 in Filesystem

Download history 130/week @ 2022-08-14 105/week @ 2022-08-21 42/week @ 2022-08-28 419/week @ 2022-09-04 113/week @ 2022-09-11 31/week @ 2022-09-18 47/week @ 2022-09-25 223/week @ 2022-10-02 167/week @ 2022-10-09 92/week @ 2022-10-16 260/week @ 2022-10-23 160/week @ 2022-10-30 157/week @ 2022-11-06 180/week @ 2022-11-13 318/week @ 2022-11-20 109/week @ 2022-11-27

764 downloads per month

MIT and GPL-3.0 licenses

290KB
7.5K SLoC

Pearl

build tests Crate Status Docs Status

Append only key-value blob storage on disk

Table of Contents

Overview

Storage scheme

pearl storage scheme

Blob

Header

Structure:

struct Header {
    magic_byte: u64,
    version: u32,
    flags: u64,
}

Description

Field Size, B Description
magic_byte 8 marks pearl blob
version any used to check compatibility
flags 1 additional file props

Record

Header

Structure:

pub struct Header {
    magic_byte: u64,
    key: Vec<u8>,
    meta_size: u64,
    data_size: u64,
    flags: u8,
    blob_offset: u64,
    created: u64,
    data_checksum: u32,
    header_checksum: u32,
}

Description

Field Size, B Description
magic_byte 8 separates records in blob
key (any) key for record location and searching
meta_size 8 meta length
data_size 8 data length (without header)
flags 1 additional record metadata
blob_offset 8 record offset from blob start
created 8 created timestamp
data_checksum 4 data crc32 checksum (without header)
header_checksum 4 header crc32 checksum (only record header)

Rust Version

pearl works on stable rust and above

$ rustup update
$ cargo build

Dependencies

~14–22MB
~479K SLoC