#distributed-systems #consensus #ha

bin+lib connect

A framework for implementing distributed systems in Rust

2 releases

0.1.1 Jun 19, 2018
0.1.0 Jun 5, 2018

#5 in #ha

23 downloads per month
Used in twirl

MIT license

97KB
2K SLoC

Connect Framework Build Status

Connect framework is primarily aimed at easing the process of developing fault-tolerant distributed systems in Rust.

What is distributed system?

A distributed system is a group of machines that are virtually or geographically seperated and that work togather to provide the same service or application to clients.

Why are distributed systems needed?

With the exponential growth in volume of highly sensitive data, it has become increasingly important to distribute and replicate it across a cluster of machines to maintain reliable and fault-tolerant systems.

There are many cases n which the use of single computer would be possible in principle but the use of distributed system is benificial for practical reasons. For example, it may be more cost efficient to obtain the desired level of performance by using a cluster of several low-end computers, in comparison with a single high-end computer. A distributed system can provide more reliability than a non-distributed system, as there is no single point of failure. Moreover, a distributed system may be easier to expand and manage than a monolithic uniprocessor system.

How distributed systems work in harmony?

A fundamental problem in distributed systems is to create harmony between different nodes of a cluster. The underlying problem requires agreement among number of nodes for a single data value.

There are many algorithms especially created for handling consensus among nodes in a cluster. Connect framework uses Raft consensus algorithm to replicate state across the cluster.

Usage

Below is a simple example showing a way of creating a distributed atomic counter using connect framework.

use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::thread;
use std::time::Duration;

use bytes::Bytes;
use connect::{Cluster, MessageType, Server, ServiceType};

let counter = Arc::new(AtomicUsize::default());

let cluster = Cluster::with_file("Connect.toml");

let incoming = |bytes: Bytes| {
    if bytes[..4] == b"incr"[..] {
        MessageType::Write
    } else {
        MessageType::Read
    }
};

let on_read = {
    let counter = counter.clone();
    move |_| Bytes::from(format!("{}\n", counter.load(Ordering::SeqCst)))
};

let on_write = {
    let counter = counter.clone();
    move |_| {
        counter.store(counter.load(Ordering::SeqCst) + 1, Ordering::SeqCst);

        Bytes::from(format!("{}\n", counter.load(Ordering::SeqCst)))
    }
};

let server = Server::new(cluster, incoming, on_read, on_write, ServiceType::InMemory);

server.run();

Important notice

This crate is under active development. Please do not use it in production.

Dependencies

~4MB
~78K SLoC