7 stable releases
new 2.4.0 | May 21, 2025 |
---|---|
2.3.0 | May 16, 2025 |
1.0.1 | May 2, 2025 |
1.0.0 | Apr 27, 2025 |
#245 in Database interfaces
895 downloads per month
245KB
5K
SLoC
RockSolid Store
RockSolid Store is a Rust library providing a robust, ergonomic, and opinionated persistence layer on top of the powerful RocksDB embedded key-value store. It aims to simplify common database interactions while offering flexibility for advanced use cases through comprehensive Column Family (CF) support, transactional operations, and advanced tuning capabilities.
Core Features
- Column Family (CF) Aware Architecture:
RocksDbCFStore
: The foundational non-transactional store for operations on any named Column Family.RocksDbStore
: A convenience wrapper for non-transactional operations targeting the default Column Family.RocksDbCFTxnStore
: The core transactional store, enabling pessimistic transactions across multiple Column Families.RocksDbTxnStore
: A convenience wrapper for transactional operations on the default Column Family.
- Flexible Configuration:
- Per-CF tuning profiles (
TuningProfile
enum for common workloads likeLatestValue
,TimeSeries
,MemorySaver
). - Customizable DB-wide and CF-specific
rocksdb::Options
via callbacks using theTunable
wrapper. - Support for native RocksDB
TransactionDBOptions
.
- Per-CF tuning profiles (
- Simplified API & Powerful Iteration:
- Ergonomic methods for Create, Read, Update, Delete (CRUD) operations via
CFOperations
andDefaultCFOperations
traits. - Flexible iteration system (
IterConfig
) supporting deserialized items, raw byte pairs, or control-only processing, with prefix and range scanning, and custom control functions.
- Ergonomic methods for Create, Read, Update, Delete (CRUD) operations via
- Automatic Serialization: Uses
serde
for values (MessagePack viarmp-serde
) and anAsBytes
trait for keys (typically raw byte sequences). - Transactional Support:
- Pessimistic transactions with CF-awareness using
RocksDbCFTxnStore
. TransactionContext
for managing operations within a default-CF transaction with a fluent API.
- Pessimistic transactions with CF-awareness using
- Atomic Batch Writes:
BatchWriter
for grouping write operations (put, delete, merge) atomically on a single, specified Column Family.- Supports multi-CF atomic batches via the
BatchWriter::raw_batch_mut()
escape hatch.
- Advanced Data Handling:
- Configurable Merge Operators: Define custom merge logic for data types.
MergeRouterBuilder
: Allows routing merge operations based on key patterns to different handler functions within the same CF. The resultingMergeOperatorConfig
is then applied to the desired CF.
- Configurable Compaction Filters:
CompactionFilterRouterBuilder
: Enables defining custom logic (remove, keep, change value) during RocksDB's compaction process, routed by key patterns. Ideal for implementing TTL, data scrubbing, or schema migrations. The resultingRockSolidCompactionFilterRouterConfig
is applied to the desired CF.
- ⚠️ Router Warning (Merge & Compaction): The routing mechanisms for both Merge Operators and Compaction Filters (via
MergeRouterBuilder
andCompactionFilterRouterBuilder
) rely on globally shared staticmatchit::Router
instances. This means routes added via any builder instance become part of a single, application-wide routing table for that type of operator.- Implication: If you configure multiple CFs (or DB instances) to use merge/compaction operators that point to these global router functions (e.g., by using the same
operator_name
in the builder and applying the built config), they will all share the same set of routes. - Recommendation: To avoid unintended route sharing or conflicts:
- Ensure key patterns used in routes are unique across all intended uses if a single global router function is used.
- Alternatively, if you need distinct routing logic for different CFs/operators, ensure you provide entirely separate merge/compaction filter functions to RocksDB for those, rather than relying on the global router for all. The router is a convenience for complex routing within one logical operator.
- Implication: If you configure multiple CFs (or DB instances) to use merge/compaction operators that point to these global router functions (e.g., by using the same
- Value Expiry Hints: Support for storing values with an explicit expiry timestamp using
ValueWithExpiry<T>
. Actual data removal based on expiry typically requires a custom compaction filter (e.g., using theCompactionFilterRouterBuilder
with a handler that checksValueWithExpiry.expire_time
). - Custom Comparators: Support for alternative key sorting (e.g., natural sort) via feature flags (
natlex_sort
,nat_sort
) configurable per CF.
- Configurable Merge Operators: Define custom merge logic for data types.
- Utilities: Database backup (checkpointing) and migration helpers.
- Comprehensive Error Handling:
StoreError
enum andStoreResult<T>
for clear error reporting. - DAO Macros: Helper macros (e.g.,
generate_dao_get_cf!
,generate_dao_put_in_txn!
) to reduce boilerplate for common data access patterns.
Getting Started
-
Installation: Add
rocksolid
to yourCargo.toml
:[dependencies] rocksolid = "X.Y.Z" # Replace X.Y.Z with the latest version from crates.io serde = { version = "1.0", features = ["derive"] } # tempfile = "3.8" # Often useful for examples and tests # env_logger = "0.11" # For enabling logging in examples/tests
For custom comparators (natural sort), enable the corresponding features:
rocksolid = { version = "X.Y.Z", features = ["natlex_sort", "nat_sort"] }
Ensure RocksDB system dependencies are installed. Refer to the
rust-rocksdb
documentation for system-specific instructions (e.g.,librocksdb-dev
,clang
,llvm
). -
Usage Guide & Examples:
- For practical examples covering various features (basic usage, CF operations, batching, transactions, merge routing, compaction filter routing, tuning), please see the files in the
examples/
directory within the repository. - Each example is a runnable
main.rs
file (e.g.,cargo run --example basic_usage
).
- For practical examples covering various features (basic usage, CF operations, batching, transactions, merge routing, compaction filter routing, tuning), please see the files in the
-
Full API Reference:
- For an exhaustive list of all public types, traits, functions, and their detailed signatures, please refer to the generated rustdoc documentation on docs.rs.
- A summary can also be found in
API_REFERENCE.md
in the repository.
Key Concepts Quick Look
- Choose your Store:
- Simple, default CF, non-transactional:
RocksDbStore
- Multiple CFs, non-transactional:
RocksDbCFStore
- Simple, default CF, transactional:
RocksDbTxnStore
- Multiple CFs, transactional:
RocksDbCFTxnStore
- Simple, default CF, non-transactional:
- Configuration is Key: Use
RocksDbCFStoreConfig
(or its simpler counterpartsRocksDbStoreConfig
,RocksDbCFTxnStoreConfig
,RocksDbTxnStoreConfig
) to define paths, CFs to open, and CF-specific settings like tuning profiles, merge operators, comparators, and compaction filters usingBaseCfConfig
orCFTxConfig
. - Batching for Atomicity/Performance: Use
batch_writer()
on your store instance. This returns aBatchWriter
scoped to a specific Column Family. - Transactions for Complex Atomicity:
RocksDbTxnStore::transaction_context()
: Returns aTransactionContext
for fluent operations on the default CF.RocksDbCFTxnStore::execute_transaction(|txn| { ... })
: For multi-CF transactional logic.
- Advanced Data Logic:
MergeRouterBuilder
: For custom conflict resolution or data aggregation based on key patterns within a CF.CompactionFilterRouterBuilder
: For background data cleanup/transformation (like TTL) based on key patterns within a CF.
Contributing
Contributions are welcome! Please feel free to open an issue to discuss bugs, feature requests, or potential improvements. Pull requests are also encouraged.
License
Licensed under the Mozilla Public License Version 2.0. See LICENSE (or the MPL-2.0 text if a separate file is not present) for details.
Dependencies
~24–33MB
~581K SLoC