entanglement

Alpha Entanglement: Rust Library for Distributed Storage. Implements alpha entanglement codes for robust error correction and data availability in distributed systems. Features an abstract storage interface for flexible protocol integration (e.g., IROH, IPFS). Optimized for fault tolerance and efficient data recovery in decentralized environments.

lib.rs:

Entanglement Storage Library

The Entanglement Storage Library provides a unified interface for interacting with various storage backends. It defines traits and implementations for uploading, downloading, and managing data in chunks. The library supports both in-memory and persistent storage solutions, making it suitable for a wide range of applications.

Modules

• iroh: Contains the implementation for the Iroh storage backend.
• storage: Defines the core traits and types for storage operations.
• mock: Provides a mock implementation of the storage traits for testing purposes.

Traits

• Storage: Represents a storage backend capable of uploading, downloading, and streaming data in chunks.
• ChunkId: A trait used to identify chunks.
• ChunkIdMapper: A trait for mapping chunk indices to chunk ids and vice versa.

Error Handling

The library uses the Error enum to represent various errors that can occur during storage operations.

Usage

To use the library, you need to implement the Storage trait for your storage backend. The library provides a default implementation for in-memory storage through the FakeStorage struct, which can be used for testing.

Example

pub mod storage;
pub mod iroh;

use crate::storage::Storage;
use crate::iroh::IrohStorage;
use bytes::Bytes;
use futures::StreamExt;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let storage = IrohStorage::new_in_memory().await?;
    let data = b"Hello, world!".to_vec();
    let upload_result = storage.upload_bytes(data.clone()).await?;

    // Download the data as a single blob
    let mut stream = storage.download_bytes(&upload_result.hash).await?;
    let mut downloaded = Vec::with_capacity(stream.size_hint().0);
    while let Some(chunk) = stream.next().await {
        downloaded.extend_from_slice(&chunk?);
    }
    assert_eq!(data, downloaded);

    // Iterate through chunks
    let mut chunk_stream = storage.iter_chunks(&upload_result.hash).await?;
    while let Some((chunk_id, chunk_result)) = chunk_stream.next().await {
        let chunk = chunk_result?;
        println!("Got chunk {}: {:?} bytes", chunk_id, chunk.len());
    }

    Ok(())
}