A drop-in replacement of tokio::time::timeout which is much more efficient.

Similar to tokio::time::timeout but more efficient on busy concurrent IOs where timeouts are created and canceled very frequently.

This crate provides the following optimizations

• The timeouts lazily initializes their timer when the Future is pending for the first time.
• There is no global lock for creating and cancelling timeouts.
• Timeout timers are rounded to the next 10ms tick and timers are shared across all timeouts with the same deadline.

Benchmark:

438.302µs total, 4ns avg per iteration

v.s. Tokio timeout():

10.716192ms total, 107ns avg per iteration

Pingora

What is Pingora

Pingora is a Rust framework to build fast, reliable and programmable networked systems.

Pingora is battle tested as it has been serving more than 40 million Internet requests per second for more than a few years.

Feature highlights

• Async Rust: fast and reliable
• HTTP 1/2 end to end proxy
• TLS over OpenSSL, BoringSSL, s2n-tls, or rustls(experimental).
• gRPC and websocket proxying
• Graceful reload
• Customizable load balancing and failover strategies
• Support for a variety of observability tools

Reasons to use Pingora

• Security is your top priority: Pingora is a more memory safe alternative for services that are written in C/C++
• Your service is performance-sensitive: Pingora is fast and efficient
• Your service requires extensive customization: The APIs Pingora proxy framework provides are highly programmable

Getting started

See our quick starting guide to see how easy it is to build a load balancer.

Our user guide covers more topics such as how to configure and run Pingora servers, as well as how to build custom HTTP servers and proxy logic on top of Pingora's framework.

API docs are also available for all the crates.

Notable crates in this workspace

• Pingora: the "public facing" crate to build networked systems and proxies
• Pingora-core: this crate defines the protocols, functionalities and basic traits
• Pingora-proxy: the logic and APIs to build HTTP proxies
• Pingora-error: the common error type used across Pingora crates
• Pingora-http: the HTTP header definitions and APIs
• Pingora-openssl & pingora-boringssl: SSL related extensions and APIs
• Pingora-ketama: the Ketama consistent algorithm
• Pingora-limits: efficient counting algorithms
• Pingora-load-balancing: load balancing algorithm extensions for pingora-proxy
• Pingora-memory-cache: Async in-memory caching with cache lock to prevent cache stampede
• Pingora-s2n: SSL extensions and APIs related to s2n-tls
• Pingora-timeout: A more efficient async timer system
• TinyUfo: The caching algorithm behind pingora-memory-cache

Note that Pingora proxy integration with caching should be considered experimental, and as such APIs related to caching are currently highly volatile.

System requirements

Systems

Linux is our tier 1 environment and main focus.

We will try our best for most code to compile for Unix environments. This is for developers and users to have an easier time developing with Pingora in Unix-like environments like macOS (though some features might be missing)

Windows support is preliminary by community's best effort only.

Both x86_64 and aarch64 architectures will be supported.

Rust version

Pingora keeps a rolling MSRV (minimum supported Rust version) policy of 6 months. This means we will accept PRs that upgrade the MSRV as long as the new Rust version used is at least 6 months old. However, we generally will not bump the highest MSRV across the workspace without a sufficiently compelling reason.

Our current MSRV is 1.84.

Currently not all crates enforce rust-version as it is possible to use some crates on lower versions.

Build Requirements

Some of the crates in this repository have dependencies on additional tools and libraries that must be satisfied in order to build them:

• Make sure that Clang is installed on your system (for boringssl)
• Make sure that Perl 5 is installed on your system (for openssl)

Contributing

Please see our contribution guidelines.

License

This project is Licensed under Apache License, Version 2.0.