1 unstable release
0.1.0 | Aug 29, 2021 |
---|
#375 in #search
46 downloads per month
Used in rosewood
11KB
186 lines
Loam
A Rust library for storing tree-like data structures to files. The intended use-case is for querying data which does not fit into RAM. The motivating project, rosewood, uses GeoSpatial data in an RTree.
Loam allows you to store anything which implements Serialize
. Data is
appended to the end of the file and never modified once written. This enables
the use of mmap
to read files without the risk of undefined behavior.
Write Example
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut writer = loam::Writer::new("../target/test.loam")?;
let id = writer.push(&"Don't forget me!")?;
writer.checkpoint(id)?;
Ok(())
}
Read Example
fn main() -> Result<(), Box<dyn std::error::Error>> {
let reader = loam::Reader::new("../target/test.loam")?;
let id = reader.root()?;
let msg: String = reader.lookup(id)?;
dbg!(msg);
Ok(())
}
File Format
A loam file starts with a Header, followed by a series of Chunks.
Header
The header is fixed-length ASCII text (8 bytes).
Field | Value | Bytes |
---|---|---|
Magic | loam |
4 |
Major Version | digits: 00 |
2 |
Minor Version | digits: 00 |
2 |
Chunks
A chunk consists of these fields, serialized using bincode:
Field | Description |
---|---|
Length | Number of bytes in Data (variable-size integer) |
Data | Serialized chunk data |
Checksum † | CRC-32 of Length + Data (fixed-size integer) |
An Id is the file offset of a chunk. It can be used to Deserialize
the
Data field.
† Checksums are only included if the crc
feature is enabled.
Checkpoint
A checkpoint is a special chunk containing a fixed-size u64
of the root
Id. A file must always end with a checkpoint, to allow readers to lookup
the root without needing to scan the entire file.
Dependencies
~0.8–1.4MB
~32K SLoC