#version-control #version #git #rocksdb #leveldb

vsdb

A stuctured DB with some 'Git-like' features, mainly used in blockchain projects

48 releases (23 breaking)

0.62.0 Feb 18, 2024
0.61.1 Oct 10, 2023
0.61.0 Jul 21, 2023
0.54.0 Mar 27, 2023
0.11.6 Dec 25, 2021

#11 in #vcs

Download history 11/week @ 2023-11-26 3/week @ 2023-12-03 6/week @ 2023-12-10 18/week @ 2023-12-17 38/week @ 2023-12-24 3/week @ 2024-01-07 1/week @ 2024-01-14 18/week @ 2024-01-21 5/week @ 2024-02-04 7/week @ 2024-02-11 145/week @ 2024-02-18 133/week @ 2024-02-25 91/week @ 2024-03-03 34/week @ 2024-03-10

405 downloads per month
Used in 8 crates (6 directly)

GPL-3.0 license

605KB
17K SLoC

GitHub top language Rust Latest Version Rust Documentation Minimum rustc version

VSDB

VSDB is a 'Git' in the form of KV-database.

Based on the powerful version control function of VSDB, you can easily give your data structure the ability to version management.

Make everything versioned !!

To view the change log check here.

Highlights

  • Support Git-like verison operations, such as:
    • Create countless branches and merge them to their parents
    • Rolling back a 'branch' to a specified historical 'version'
    • Querying the historical value of a key on the specified 'branch'
  • Most APIs is similar as the coresponding data structures in the standard library
    • Use Vecx just like Vec
    • Use Mapx just like HashMap
    • Use MapxOrd just like BTreeMap
  • ...

Practical scenarios

  • Process CheckTx, DeliverTx, Commit ... in 'Tendermint ABCI'
  • Handle folk chain branches, e.g.:
    • Handling 'chain folk' or 'uncle blocks' in non-deterministic consensus like 'POW'
    • Handle temporary 'chain folk' in a hybrid consensus like 'Babe + Grandpa'(substrate)
  • Support some special APIs of 'ETH Web3' in the form of 'trial run'
  • ...

Examples

Suppose you have a great algorithm like this:

struct GreatAlgo {
    a: Vec<...>,
    b: BTreeMap<...>,
    c: u128,
    d: HashMap<...>,
    e: ...
}

Simply replace the original structure with the corresponding VSDB data structure, and your algorithm get the powerful version control ability at once!

#[dervive(Vs, Default)]
struct GreatAlgo {
    a: VecxVs<...>,
    b: MapxOrdVs<...>,
    c: OrphanVs<u128>,
    d: MapxVs<...>,
    e: ...
}

let algo = GreatAlgo::default();

algo.get_by_branch_version(...);
algo.branch_create(...);
algo.branch_create_by_base_branch(...);
algo.branch_create_by_base_branch_version(...);
algo.branch_remove(...);
algo.version_pop(...);
algo.prune();

NOTE !!

the #[derive(Vs)] macro can be applied to structures whose internal fields are all types defined in VSDB (primitive types and their collections are also supported), but can not be applied to nesting wrapper among VSDB-types, we recommend you to use the multi-key APIs if you indeed require these functions(better performance also), or you will have to implement the VsMgmt trait manually.

This data structure can be handled correctly by #[derive(Vs)]:

#[derive(Vs)]
struct GoodCase<K, T> {
    a: VecxVs<i64>,
    b: SubItem0,
    c: SubItem1,
    d: SubItem2,
    e: u8,
    f: Vec<i16>,
    g: VecDeque<i64>,
    h: BTreeSet<u16>,
    i: HashMap<K, AtomicU64>,
    j: HashSet<i32>,
    k: LinkedList<()>,
    l: Box<dyn AsRef<bool>,
    m: Box<dyn AsRef<[Vec<u128>]>>,
    n: PhantomData<T>,
}

#[derive(Vs)]
struct SubItem0(MapxVs<u8, u8>, VecxVs<u8>);

#[derive(Vs)]
struct SubItem1 {
    a: OrphanVs<i16>,
    b: MapxOrdVs<String, u8>
}

#[derive(Vs)]
struct SubItem2 {
    a: i8,
    b: u128
}

// // A nope implementation of `VsMgmt` for custom stateless types.
// // the `#[derive(Vs)]` on 'SubItem2' is same as this implementation.
// impl VsMgmt for SubItem2 {
//     impl_vs_methods_nope!();
// }

But this one can NOT be handled correctly by #[derive(Vs)]:

// It can be compiled, but the result is wrong !
// The versioned methods of the inner 'MapxVs<u8, u8>' will missing,
// We recommend you to use the 'multi-key' APIs of VSDB, or
// you will have to implement the 'VsMgmt' trait manually.
#[derive(Vs)]
struct BadCase {
    a: VecxVs<MapxVs<u8, u8>>,
}

Please check the multi-key functions if you have requirements of the above or similar scenes.

Some complete examples:

Compilation features

  • [ DEFAULT ] rocks_backend, use rocksdb as the backend database
    • Stable
    • C++ implementation, difficult to be compiled into a static binary
  • parity_backend, use parity-db as the backend database
    • Experimental
    • Pure rust implementation, can be easily compiled into a static binary
  • bcs_codec, use bcs as the codec
    • Faster running speed than json
    • Security reinforcement for blockchain scenarios
    • Created by the original 'Libre' project of Facebook
  • msgpack_codec, use rmp-serde as the codec
    • Faster running speed than json
  • json_codec, use serde_json as the codec
    • Better generality and compatibility
  • compress, enable compression in the backend database
  • extra_types, implement VsMgmt for some common extra types
    • For example: H256 and H160 of the primitive-types crate
  • vs, enable all versioned APIs and the Vs procedural macro

NOTE

  • The serialized result of a VSDB instance can not be used as the basis for distributed consensus
    • The serialized result only contains some meta-information(storage paths, etc.)
    • These meta-information are likely to be different in different environments
    • The correct way is to read what you need from it, and then process the real content
  • Version names must be globally unique
    • Using a same version name on different branches is also not allowed
  • The low-level "does not exist" expression in MapxRawVs and MapxRawMkVs has been changed from None to [] since v0.42.0
    • If you assign an empty value([u8;0], &[] ...) to a key, the key will be treated as 'deleted'
    • NOTE: these 'empty' values can be set as normal value in other instances except this two!

LICENSE

  • MIT for v0.40 and earlier
  • GPL-3.0 for v0.41 and later

Dependencies

~2–15MB
~179K SLoC