16 releases

0.6.0 Dec 22, 2024
0.5.12 Nov 23, 2024
0.5.11 Oct 15, 2024
0.5.8 Aug 6, 2024
0.4.9 Feb 7, 2024

#330 in Concurrency

Download history 3/week @ 2024-09-23 246/week @ 2024-09-30 86/week @ 2024-10-07 207/week @ 2024-10-14 44/week @ 2024-10-21 62/week @ 2024-10-28 36/week @ 2024-11-04 325/week @ 2024-11-11 473/week @ 2024-11-18 145/week @ 2024-11-25 189/week @ 2024-12-02 184/week @ 2024-12-09 332/week @ 2024-12-16 195/week @ 2024-12-23 158/week @ 2024-12-30 140/week @ 2025-01-06

827 downloads per month
Used in 4 crates

ISC license

48KB
1K SLoC

smolscale

A global, auto-scaling scheduler for [async-task] using work-balancing.

What? Another executor?

smolscale is a work-balancing executor based on [async-task], designed to be a drop-in replacement to smol and async-global-executor. It is designed based on the idea that work-stealing, the usual approach in async executors like async-executor and tokio, is not the right algorithm for scheduling huge amounts of tiny, interdependent work units, which are what message-passing futures end up being. Instead, smolscale uses work-balancing, an approach also found in Erlang, where a global "balancer" thread periodically balances work between workers, but workers do not attempt to steal tasks from each other. This avoids the extremely frequent stealing attempts that work-stealing schedulers generate when applied to async tasks.

smolscale's approach especially excels in two circumstances:

  • When the CPU cores are not fully loaded: Traditional work stealing optimizes for the case where most workers have work to do, which is only the case in fully-loaded scenarios. When workers often wake up and go back to sleep, however, a lot of CPU time is wasted stealing work. smolscale will instead drastically reduce CPU usage in these circumstances --- a async-executor app that takes 80% of CPU time may now take only 20%. Although this does not improve fully-loaded throughput, it significantly reduces power consumption and does increase throughput in circumstances where multiple thread pools compete for CPU time.
  • When a lot of message-passing is happening: Message-passing workloads often involve tasks quickly waking up and going back to sleep. In a work-stealing scheduler, this again floods the scheduler with stealing requests. smolscale can significantly improve throughput, especially compared to executors like async-executor that do not special-case message passing.

License: ISC

Dependencies

~11–40MB
~623K SLoC