#in-memory #database-schema #durable #storage #page #high-concurrency

darkbird

HighConcurrency, RealTime, InMemory storage inspired by erlang mnesia

27 stable releases (5 major)

6.2.4 Dec 21, 2023
6.2.0 Oct 27, 2023
6.1.8 Jul 21, 2023
6.1.0 Feb 18, 2023
1.1.2 Jul 5, 2022

#452 in Database interfaces

Download history 7/week @ 2023-12-22 15/week @ 2024-01-05 1/week @ 2024-01-12 20/week @ 2024-02-16 46/week @ 2024-02-23 24/week @ 2024-03-01 31/week @ 2024-03-08 14/week @ 2024-03-15 15/week @ 2024-03-22 38/week @ 2024-03-29 33/week @ 2024-04-05

116 downloads per month
Used in http-cache-darkbird

Apache-2.0

160KB
4K SLoC

DarkBird

DarkBird is a document-oriented, real-time, in-memory database solution optimized for fast document retrieval.

Features

  • Database level: darkbird was storage, but from ( vsn 5.0.3 ) is full-featured database because provide Schema for building database and all operation do with database layer
  • Persistent: Uses a non-blocking write-ahead-logging engine for data persistence, storing data to multiple pages.
  • In-memory: Data is stored in memory, with two modes (DiskCopies, RamCopies), the first persisting data to disk and reloading the data into memory after restart.
  • Concurrency: Uses a high-concurrent HashMap (DashMap) and doesn't require Mutex/RwLock for thread synchronization.
  • Vector: darkbird provide a vector engine for storing and searching vectors
  • Indexing: Supports indexing, allowing for dynamic decision-making about which document fields to index.
  • Full-text search: Supports full-text search operations since version 3.5.0.
  • Materialized view: Supports materialized view
  • Tagging: Each document can have multiple tags, and one tag can refer to many documents, making it great for indexing groups of documents for fast retrieval by key.
  • Expiration: from vsn 6.0.0 support key expiry.
  • Atomic Operation: from vsn 6.0.0 support Atomic Operation (just like redis setNx)
  • Migration: The storage model is (Key, Document), and you can use migration to change the existing (Key, Document) data on disk before opening the storage.
  • Backup / Restore from vsn-6.1.0 support Backup/Restore
  • External database support: Copy storage data to Postgres/Cassandra/Scylla and load from it.
  • Event handling: Subscribe to any channel to receive events.

Crate

darkbird = "6.2.4"

Examples

  • See the complete examples here.
  • This repo is Movies store service with (darkbird + actix-web) here.

Versions

  • 2.0.0: Improved performance and added persistent copy of whole data to a database.
  • 3.0.0: Implemented indexing, tagging, and range queries. Document model must implement tree trait from this version
  • 3.5.0: Added full-text search API
  • 4.0.0: Added materialized view support. Document models must implement the MaterializedView trait, and API is provided to fetch view models. Uses &str instead of &String for better performance and API compatibility. All examples are updated.
  • 5.0.1: migrated from Storage to Database world with Schema builder and Database layer to do all operation also is compatible with older version
  • 5.0.2: fixedbug persist indexing and changed fullText search api for a bug all examples updates
  • 5.0.3: fixedbug loader
  • 6.0.0: added another storage Engine for supporting: atomic operation (just like redis setNx), expiration and simpler api
  • 6.0.1: Backup/Restore _ new migration component (recover self if occure error)
  • 6.2.0: Vector Engine

Buy Me A Coffee

Dependencies

~14–27MB
~441K SLoC