#rocks-db #facebook #rocksdb #ffi

sys rocks-sys

Raw RocksDB bindings for Rust, used internally in rust-rocks

19 releases

0.1.10 Jan 16, 2021
0.1.9 Aug 8, 2020
0.1.8 May 28, 2020
0.1.5 Mar 22, 2020
0.0.8 Jun 27, 2017

#1620 in Database interfaces

32 downloads per month
Used in rocks

Apache-2.0

13MB
281K SLoC

C++ 263K SLoC // 0.1% comments Python 7K SLoC // 0.2% comments Rust 4K SLoC // 0.0% comments Shell 3K SLoC // 0.2% comments C 2.5K SLoC // 0.0% comments GNU Style Assembly 539 SLoC // 0.1% comments INI 433 SLoC // 0.1% comments PowerShell 368 SLoC // 0.2% comments Bitbake 168 SLoC // 0.2% comments JavaScript 95 SLoC // 0.1% comments Bazel 41 SLoC // 0.2% comments

RustRocks

crates.io badge DOCS.RS badge Linux Build Status macOS Build Status Windows Build Status

Another RocksDB binding for Rust. Documentation

Make RocksDB really rocks!

  • Static link against RocksDB 6.7.3 (git submodules)
  • Dynamic link tested:
    • macOS homebrew
    • Windows 10, VS 2019 with vcpkg
    • ArchLinux pacman, both x86_64 and aarch64(ODroid-C2)
    • Ubuntu 18.04 (rocksdb5.8 branch), both x86_64 and aarch64(RPi 3)
    • Ubuntu 20.04 (rocksdb5.17 branch)

Installation

Dynamicly link RocksDB:

[dependencies]
rocks = "0.1"

Static link against RocksDB(with snappy enabled by default):

[dependencies.rocks]
version = "0.1"
default-features = false
features = ["static-link"]

With all static features(all compression types):

[dependencies.rocks]
version = "0.1"
default-features = false
features = ["full"]

How to compile

Feel free to refer Travic-CI, AppVeyor and Github Actions configuration files.

$ git submodule update --init --recursive
$ cargo test --features static-link -- --test-threads 1
(This will build with snappy as the only compression supported)

$ cargo test --features full -- --test-threads 1
(This will build with all compression supported)

For macOS(with RocksDB installed via brew):

$ brew install rocksdb
$ cargo test -- --nocapture --test-threads 1

For Linux:

(install rocksdb via package manager or make & install)
$ sudo apt install lld
(NOTE: gcc-ld can't handle circular references while linking.)
(for more, refer the last section of README)
$ RUSTFLAGS="-C link-arg=-fuse-ld=lld" cargo test -- --test-threads 1

Use environment variables if rocksdb is installed to non-default directory:

LD_LIBRARY_PATH=/usr/local/lib LIBRARY_PATH=/usr/local/lib CXXFLAGS=-I/usr/local/include

Ubuntu LTS

RocksDB changes its API often, so rust-rocks use different branch to support Ubuntu LTS.

> sudo apt install librocksdb-dev libsnappy-dev

You also need lld form official source or http://apt.llvm.org/.

Branches:

  • rocksdb5.8 (18.04 LTS)
  • rocksdb5.17 (20.04 LTS)

Windows

You need VS 2017 or VS 2019, and install RocksDB via vcpkg.

FAQ

Feel free to Open a New Issue.

List current supported compression types

$ cargo run --example it-works
RocksDB: 6.7.3
Compression Supported:
  - NoCompression
  - SnappyCompression
  - ZlibCompression
  - BZip2Compression
  - LZ4Compression
  - LZ4HCCompression
  - ZSTD
  - ZSTDNotFinalCompression

Development

Bindgen:

$ cd rocks-sys
$ PATH="/usr/local/opt/llvm/bin:$PATH" make
(this will regenerate the bindgen c.rs)

Known bugs

Linking error under Linux

  • rust-rocks exports rust functions to c++, so there are circular references while linking
  • GCC requires that you put the object files and libraries in the order that they depend on each other
  • Rust will not wrap user crates in --start-group and --end-group
  • So circular references will be errors.
  • Can be fixed by using lld as linker, RUSTFLAGS="-C link-arg=-fuse-ld=lld"
  • Can be fixed by manually organising link arguments
    • librocks, then librocks_sys, then librocks again

Minor memory leaks

  • The raw pointers are created on the fly, should be impled via lazy_static and wrapped in trait objects
    • ColumnFamilyOptions::comparator: const Comparator*
    • ColumnFamilyOptions::compaction_filter: const CompactionFilter*

Iterator leaks lifetime

Ref: https://github.com/bh1xuw/rust-rocks/issues/15

  • While doing a for-traverse: That's OK
  • While collecting for later use: Clone the keys and values

No runtime deps