#task-queue #thread-pool #multi-threading #abstraction #composable #flexible #offers

task_pool

Flexible abstraction for task-based composable multithreading

5 releases

0.1.4 Dec 24, 2023
0.1.3 Nov 13, 2023
0.1.2 Nov 6, 2023
0.1.1 Aug 1, 2023
0.1.0 Jul 28, 2023

#287 in Concurrency

Download history 5/week @ 2023-12-22 6/week @ 2024-02-23 2/week @ 2024-03-01 7/week @ 2024-03-15 7/week @ 2024-03-22 9/week @ 2024-03-29 106/week @ 2024-04-05

129 downloads per month
Used in datafrost

MIT/Apache

42KB
816 lines

task_pool

Crates.io Docs.rs

task_pool offers a flexible abstraction for composing and distributing work within a fixed hardware threadpool. To that end, it offers the following features:

  • The ability to define and compose sources of work
  • The ability to create hardware threadpool and consume those sources
  • A variety of high-level abstractions for scheduling, such as awaitable tasks

Usage

To use task_pool, there are three steps:

  1. Creating and initializing WorkProvider instances (such as a queue or chain of multiple queues)
  2. Creating a hardware TaskPool which consumes those instances
  3. Spawning high-level tasks on the WorkProviders which are handled by the threadpool

The following example shows these steps in action:

// 1. Create a queue from which we can spawn tasks
let queue = TaskQueue::<Fifo>::default();

// 2. Create a threadpool that draws from the provided queue. Forget the threadpool so that it runs indefinitely.
TaskPool::new(queue.clone(), 4).forget();

// 3. Spawn a task into the queue and synchronously await its completion.
assert_eq!(queue.spawn(once(|| { println!("This will execute on background thread."); 2 })).join(), 2);

// ...or, asynchronously await its completion.
assert_eq!(queue.spawn(once(|| { println!("This will execute on background thread."); 2 })).await, 2);

Dependencies

~1.4–2.8MB
~49K SLoC