#partitioning #task #worker #hash #count #sets #data

task_partitioner

helper lib that allows partitioning named tasks by count or hash

2 releases

0.1.1 Nov 25, 2022
0.1.0 Nov 25, 2022

#8 in #partitioning

Download history 191/week @ 2024-01-04 63/week @ 2024-01-11 50/week @ 2024-01-18 61/week @ 2024-01-25 54/week @ 2024-02-01 92/week @ 2024-02-08 80/week @ 2024-02-15 92/week @ 2024-02-22 66/week @ 2024-02-29 62/week @ 2024-03-07 64/week @ 2024-03-14 80/week @ 2024-03-21 74/week @ 2024-03-28 130/week @ 2024-04-04 54/week @ 2024-04-11 28/week @ 2024-04-18

292 downloads per month
Used in laze

MIT/Apache

10KB
232 lines

Introduction

This Rust library allows partitioning a list of tasks to multiple workers, by count or by hash. It was cut out from the awesome nextest.

License

task_partitioner is licensed under the terms of the Apache License (Version 2.0), or MIT.


lib.rs:

Support for partitioning task runs across several machines.

At the moment this only supports simple hash-based and count-based sharding. In the future it could potentially be made smarter: e.g. using data to pick different sets of binaries and tests to run, with an aim to minimize total build and test times.

Dependencies

~0.5–1MB
~21K SLoC