#embedded-database #key-value-store #local-storage #cannyls

cannyls

Embedded persistent key-value storage optimized for random-access workload and huge-capacity HDD

5 releases

Uses old Rust 2015

0.10.0 Aug 7, 2020
0.9.4 Feb 25, 2020
0.9.2 Jan 16, 2019
0.9.1 Oct 26, 2018
0.9.0 Oct 17, 2018

#109 in Database implementations

Download history 74/week @ 2023-11-06 43/week @ 2023-11-13 51/week @ 2023-11-20 76/week @ 2023-11-27 23/week @ 2023-12-04 45/week @ 2023-12-11 47/week @ 2023-12-18 54/week @ 2023-12-25 28/week @ 2024-01-01 51/week @ 2024-01-08 43/week @ 2024-01-15 29/week @ 2024-01-22 42/week @ 2024-01-29 43/week @ 2024-02-05 65/week @ 2024-02-12 152/week @ 2024-02-19

302 downloads per month
Used in 8 crates

MIT license

345KB
6.5K SLoC

cannyls

Crates.io: cannyls Documentation Build Status License: MIT

CannyLS is an embedded and persistent key-value storage optimized for random-access workload and huge-capacity HDD.

CannyLS mainly has following features:

  • A local storage for storing objects that called as "lump":
    • Basically, a lump is a simple key-value entry
    • The distinctive properties are that the key is fixed length (128 bits) and suited for storing a relatively large size value (e.g., several MB)
  • Provides simple functionalities:
  • Optimized for random-access workload on huge-capacity HDD (up to 512 TB):
  • Aiming to provide predictable and stable read/write latency:
    • There are (nearly) strict upper bounds about the number of disk accesses issued when executing operations
      • One disk access when PUT and DELETE, and two when PUT
    • There are no background processings like compaction and stop-the-world GC which may block normal operations for a long time
    • For eliminating overhead and uncertainty, CannyLS has no caching layer:
      • It uses Direct I/O for bypassing OS layer caching (e.g., page cache)
      • If you need any caching layer, it is your responsibility to implement it
  • Detailed metrics are exposed using Prometheus

See Wiki for more details about CannyLS.

Documentation

Dependencies

~4MB
~76K SLoC