#partitioning #task #worker #hash #count #sets #data

task_partitioner

helper lib that allows partitioning named tasks by count or hash

2 releases

0.1.1 Nov 25, 2022
0.1.0 Nov 25, 2022

#8 in #partitioning

Download history 63/week @ 2024-07-21 95/week @ 2024-07-28 56/week @ 2024-08-04 71/week @ 2024-08-11 49/week @ 2024-08-18 57/week @ 2024-08-25 72/week @ 2024-09-01 96/week @ 2024-09-08 93/week @ 2024-09-15 77/week @ 2024-09-22 79/week @ 2024-09-29 57/week @ 2024-10-06 63/week @ 2024-10-13 61/week @ 2024-10-20 41/week @ 2024-10-27 38/week @ 2024-11-03

207 downloads per month
Used in laze

MIT/Apache

10KB
232 lines

Introduction

This Rust library allows partitioning a list of tasks to multiple workers, by count or by hash. It was cut out from the awesome nextest.

License

task_partitioner is licensed under the terms of the Apache License (Version 2.0), or MIT.


lib.rs:

Support for partitioning task runs across several machines.

At the moment this only supports simple hash-based and count-based sharding. In the future it could potentially be made smarter: e.g. using data to pick different sets of binaries and tests to run, with an aim to minimize total build and test times.

Dependencies

~0.4–0.9MB
~20K SLoC