6 releases (breaking)

new 0.20.0 Jul 9, 2024
0.19.0 Jun 27, 2024
0.18.0 Jun 7, 2024
0.17.0 May 24, 2024
0.16.2 May 13, 2024

#468 in Database interfaces

Download history 319/week @ 2024-05-10 72/week @ 2024-05-17 260/week @ 2024-05-24 157/week @ 2024-05-31 313/week @ 2024-06-07 288/week @ 2024-06-14 180/week @ 2024-06-21 51/week @ 2024-06-28

840 downloads per month
Used in 3 crates (via iroh)

MIT/Apache

1MB
21K SLoC

iroh-docs

Multi-dimensional key-value documents with an efficient synchronization protocol.

The crate operates on Replicas. A replica contains an unlimited number of Entries. Each entry is identified by a key, its author, and the replica's namespace. Its value is the 32-byte BLAKE3 hash of the entry's content data, the size of this content data, and a timestamp. The content data itself is not stored or transferred through a replica.

All entries in a replica are signed with two keypairs:

  • The Namespace key, as a token of write capability. The public key is the NamespaceId, which also serves as the unique identifier for a replica.
  • The Author key, as a proof of authorship. Any number of authors may be created, and their semantic meaning is application-specific. The public key of an author is the [AuthorId].

Replicas can be synchronized between peers by exchanging messages. The synchronization algorithm is based on a technique called range-based set reconciliation, based on this paper by Aljoscha Meyer:

Range-based set reconciliation is a simple approach to efficiently compute the union of two sets over a network, based on recursively partitioning the sets and comparing fingerprints of the partitions to probabilistically detect whether a partition requires further work.

The crate exposes a generic storage interface with in-memory and persistent, file-based implementations. The latter makes use of redb, an embedded key-value store, and persists the whole store with all replicas to a single file.

License

This project is licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Dependencies

~12–51MB
~852K SLoC