PhotonDB

A high-performance storage engine for modern hardware and platforms.

PhotonDB is designed from scratch to leverage the power of modern multi-core chips, storage devices, operating systems, and programming languages.

Features:

• Latch-free data structures, scale to many cores.
• Log-structured persistent stores, optimized for flash storage.
• Asynchronous APIs and efficient file IO, powered by io_uring on Linux.

Design

Progress

We have published the photondb crate v0.0.4. You can try some examples to see what it can do so far. It is important to note that the current version is still too young to be used for anything serious.

Use the synchronous APIs:

[dependencies]
photondb = "0.0.4"

use photondb::{std::Table, Result, TableOptions};

fn main() -> Result<()> {
    let table = Table::open("/tmp/photondb", TableOptions::default())?;
    let key = vec![1];
    let val1 = vec![2];
    let val2 = vec![3];
    // Simple CRUD operations.
    table.put(&key, 1, &val1)?;
    table.delete(&key, 2)?;
    table.put(&key, 3, &val2)?;
    assert_eq!(table.get(&key, 1)?, Some(val1));
    assert_eq!(table.get(&key, 2)?, None);
    assert_eq!(table.get(&key, 3)?, Some(val2.clone()));
    let guard = table.pin();
    // Get the value without copy.
    assert_eq!(guard.get(&key, 3)?, Some(val2.as_slice()));
    // Iterate the tree page by page.
    let mut pages = guard.pages();
    while let Some(page) = pages.next()? {
        for (k, v) in page {
            println!("{:?} {:?}", k, v);
        }
    }
    Ok(())
}

Use the asynchronous APIs:

[dependencies]
photondb = "0.0.4"
photonio = "0.0.5"

use photondb::{Result, Table, TableOptions};

#[photonio::main]
async fn main() -> Result<()> {
    let table = Table::open("/tmp/photondb", TableOptions::default()).await?;
    let key = vec![1];
    let val1 = vec![2];
    let val2 = vec![3];
    // Simple CRUD operations.
    table.put(&key, 1, &val1).await?;
    table.delete(&key, 2).await?;
    table.put(&key, 3, &val2).await?;
    assert_eq!(table.get(&key, 1).await?, Some(val1.clone()));
    assert_eq!(table.get(&key, 2).await?, None);
    assert_eq!(table.get(&key, 3).await?, Some(val2.clone()));
    let guard = table.pin();
    // Get the value without copy.
    assert_eq!(guard.get(&key, 3).await?, Some(val2.as_slice()));
    // Iterate the tree page by page.
    let mut pages = guard.pages();
    while let Some(page) = pages.next().await? {
        for (k, v) in page {
            println!("{:?} {:?}", k, v);
        }
    }
    Ok(())
}

Run the example with:

cargo +nightly-2022-10-01 run

lib.rs:

A high-performance storage engine for modern hardware and platforms.

PhotonDB is designed from scratch to leverage the power of modern multi-core chips, storage devices, and operating systems.

Features:

• Latch-free data structures, scale to many cores.
• Log-structured persistent stores, optimized for flash storage.
• Asynchronous APIs and efficient file IO, powered by io_uring on Linux.

This crate provides three sets of APIs:

• Raw: a set of low-level APIs that can run with different environments.
• Std: a set of synchronous APIs based on the raw one that doesn't require a runtime to run.
• Photon: a set of asynchronous APIs based on the raw one that must run with the PhotonIO runtime.

The Photon APIs are the default and are re-exported to the top-level module for convenience.