Loam

A Rust library for storing tree-like data structures to files. The intended use-case is for querying data which does not fit into RAM. The motivating project, rosewood, uses GeoSpatial data in an RTree.

Loam allows you to store anything which implements Serialize. Data is appended to the end of the file and never modified once written. This enables the use of mmap to read files without the risk of undefined behavior.

Write Example

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut writer = loam::Writer::new("../target/test.loam")?;
    let id = writer.push(&"Don't forget me!")?;
    writer.checkpoint(id)?;
    Ok(())
}

Read Example

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reader = loam::Reader::new("../target/test.loam")?;
    let id = reader.root()?;
    let msg: String = reader.lookup(id)?;
    dbg!(msg);
    Ok(())
}

File Format

A loam file starts with a Header, followed by a series of Chunks.

Header

The header is fixed-length ASCII text (8 bytes).

Chunks

A chunk consists of these fields, serialized using bincode:

An Id is the file offset of a chunk. It can be used to Deserialize the Data field.

† Checksums are only included if the crc feature is enabled.

Checkpoint

A checkpoint is a special chunk containing a fixed-size u64 of the root Id. A file must always end with a checkpoint, to allow readers to lookup the root without needing to scan the entire file.