Mycelium D.D.M.

Mycelium Decentralized Data Mesh's goal is to be a decentralized data storage that creates a mesh of data on multiple devices like phones, tablets, and pc that can be configured to share and copy data between devices in useful ways.

• This project is being shaped from a learning project to a portfolio project. In the distant future a functional database. We have a blog, wiki page, code documentation, CI builds, and a nice code repository. Mycelium is a p2p distributed database project. Currently Docker-Compose is being used to test basic network functionality. Primarily because it was fast and simple. Once that is done we will be using something like ns-3.

Where are we now?

Mycelium Core uses a basic key/bucket store. Those buckets of data are called tags. This core data structure of the Mycelium ecosystem uses the Crossbeam library. Mycelium currently operates as a memory database but can write itself to disk. Currently this operation is slow and works on top of the OS file system.

Mycelium Index also uses Crossbeam's SkipLMS structures to index back into Core. Mycelium Index will also create relationships between items in different tags. In the long term, I want to use ML to replace indexes (pdf)..

We have a functional TCP stack, but not p2p networking. UDP multicast will create a mesh of network nodes. Mycelium's goal is a mesh of nodes will configure themselves as they come online over a LAN. Mycelium will distribute data over nodes based on the use of the data and some configuration. We will move from Tokio to rust-libp2p with Tokio as an executor in the furture after some rework to support async/await.

User Documents

Mycelium Core

Data IO library.

Mycelium Index

Mycelium Index will create relational database functionality. Data is stored in a compact binary format. To create relationships each piece of data can be given one or many index terms. Later relationships will be added allowing you to associate data. Eventually joining/filtering and operating over these indexes. Indexes can contain any data contained within a node that can be public.

Mycelium

Web server:

TCP normal requests between nodes.

UDP will be used to register nodes P2P using UDP local multicast. Will also be used for general requests over the mesh.

Mycelium lib

Primary library for embedding Mycelium into another application.

Usage:

Currently usage is not ergonomic. Packages are structured inside a single workspace. Eventually when further along project will be split out into multiple libraries, executables, and tools.

• Clonse this repository
• Create porject
• Use path to lib_mycelium in new project cargo.toml

[dependencies]
lib_mycelium = { path = "../myceliumdds/lib_mycelium" }

main.rs

extern crate lib_mycelium;

// Start a TCP Server default 127.0.0.1:34120
let config = Config::default();
let db = Mycelium::init_db(config);
mycelium_lib::start_local(db.clone());

// TCP Connect and run:
// msql insert [0, 0, 0, 0] doctor
    // insert byte[] into the database container doctor. Statement creates container if it does
    // not currently exists. 
// msql select doctor
    // selects all items that have been inserted into the container doctor

Alternatively TCP server is not required:

let config = Config::default();
let db = Mycelium::init_db(config);
let cmd_txt = format!("Insert {} duck", "Quack Quack!".as_bytes());
let cmd = Command::parse(cmd_txt.as_str()).unwrap();    
match db.execute_command(cmd).unwrap() {
    (Result::Id(id), None, None) => {
        println!("ID of inserted item: {:?}", id)
    }
    _ => { unimplemented!() }
}

// or a list of results
let cmd = Command::parse("Select from duck").unwrap();
match db.execute_command(cmd).unwrap() {
    (Result::List, None, Some(list)) => {
        for item in list {
            println!("item: {:?}", item)
        }
    },
    _  => { unimplemented!() }
}

Integrations

Project used to test the usability and cohesiveness of developed libraries.

Benchmark

Library to track and chart function performance over time.

Installing/using Mycelium D.D.S.

Install rust . Clone this repository. Build and run using cargo.

Additional Documentation

Additional documentation for development can be found by running "cargo doc" after cloning this repository.

Contributing!

I would welcome any contributions to this project. Project and all contributions will be under the MIT license.