lyquor-network: a modern network abstraction and tooling for building distributed systems.

Why not libp2p? (and others)

We believe our design is clutter-free, minimalist, and usable at different levels. It does what other libraries like libp2p can do, but with much better abstraction based on the authors' multi-year of cutting-edge research experience in distributed systems and consensus. It is not just a code base that "works", but also meets the demands of a wide spectrum of networked infrastructures, especially those requiring high volume of binary (short or moderately sized) messages. In fact, we build this library to be reused and serve well as the basis for any core infrastructure that exchanges messages in a networked manner.

Why not Tokio?

We do use tokio to manage some intertwined asynchronous logic in our code, but our core data transmission is done by a carefully crafted engine directly based on mio. Instead of throwing all connections into tokio, by giving them a better abstraction (Endpoint) and drive them through our well-engineered Driver (which is a pool of Workers), we have established a good separation among the following three aspects of a networking library: the notion of a peer-wise data transmission channel, the actual underlying transportation protocol (could even be SMS, if one implements Transport trait) it utilizes, and the execution logic that efficiently drives the asynchronous I/O induced by the transmission. By learning from the past projects and with the help of Rust, we believe our abstraction achieves clarity, simplicity and low overhead at the same time. If not the best in terms of the absolute performance (one can always handcraft a super adhoc solution, say in C, with optimal performance), it still offers a solid basis for any distributed systems with great performance and robustness.

Reusable at Different Levels

Our design deliberately divides our abstraction/implementation into different levels so engineers and researchers can easily use them for their own purpose with a lot of freedom of customization.

At the highest level, one can either use:

• network (TBD): will contain the code for self-organizing network, including node discovery, routing, etc.
• hub: defines and implements a multi-transport "hub" that could automatically connect to other peers given their contacts. This could be useful for a relatively static network setup, such as a quorum-based (or "vote-based", "PoS") consensus protocol, or MPC projects.

In hub, there are three important abstractions (and their implementations) could be customized:

• Driver: a pool of Workers, where each worker has an OS-level thread and drives the I/O of multiple endpoints.

• Endpoint: a user-friendly, bidirectional, virtual channel that transmits data to some remote recipient. An endpoint has a pluggable Transport at its core that actually implements the network transmission.

• Transport: a protocol-specific implementation that follows a unified interface to move data through the endpoint. The existence of this abstraction allows Endpoint to be connection-agnostic: the underlying transportation method could either be "connection-based" (such as TCP) or "connectionless" (such as UDP), but the user of endpoints does not need to worry about that. An endpoint may renew its underlying transport to be able to resume its data transmission once it is "clogged" (i.e., reconnection) -- all these, are invisible to the endpoint users. What's more, while doing all these, endpoints can be moved among the workers (for load-balancing purposes) on-the-fly, without losing any data.

Portability

The vast majority of our code base is also WASM-compatible. Which means one can turn web browsers and web-like environments (such as mobile apps) into a legit node that talks to others. This means one can achieve "true" P2P by running a lightweight node in the client to talk to "servers" (nodes with useful services), without having to invoke a cumbersome RPC protocol like JSON/RPC or web REST calls.

License

This project is licensed under Apache License 2.0. See NOTICE file for more information.