10 unstable releases (3 breaking)
new 0.4.5 | Oct 22, 2024 |
---|---|
0.4.4 | Oct 21, 2024 |
0.4.0 | Sep 23, 2024 |
0.3.0 | Jun 27, 2024 |
0.1.0 | Nov 16, 2023 |
#362 in Asynchronous
727 downloads per month
54KB
943 lines
Splaycast
For duplicating streams, like a broadcast.
Compared to the more general purpose and excellent tokio::sync::broadcast
,
Splaycast is more application-specific and more amenable to huge numbers of
subscribers.
About
This tool was developed to solve a problem with high contention observed on
a tokio::sync::broadcast
with 10,000 - 200,000 subscribers. It was
manifesting as slow polls in Tokio metrics, and extremely high latency for
Receivers to get their messages - hundreds of milliseconds or much worse.
Benchmarks
I've implemented a benchmark core that is generic over both Broadcast and
Splaycast, to test the differences between them in different scenarios -
varying queue depth, subscriber count, and thread count. At very low
subscriber counts, tokio::sync::broadcast
is still the better choice.
There is an inflection point at about 128 subscribers where splaycast begins
to pull away, and by 16384 subscribers the difference is 12.27ms versus on
Splaycast versus 43.6ms on tokio::sync::broadcast
. These tests were done
on a 64gb M1 laptop, so while relative differences are relevant, you should
be skeptical of absolute measurements.
With queue depth=4, "input size" along X is 4 times subscriber count. You can see that as subscriber counts grow, the behavior of Splaycast is linear with subscribers, but with a much lower constant factor than broadcast.
If we look closer at the bottom of the chart we can see the behavior more explicitly. Here with the same queue depth=4 and "input size" means 4x subscriber count, you can see that at or below 256 subscribers it doesn't make much of a difference which Stream splitter you use. But at 512 and greater subscriber counts, the difference is increasingly pronounced.
Dependencies
~1.5MB
~21K SLoC