#partitioning #task #worker #hash #count #sets #data

task_partitioner

helper lib that allows partitioning named tasks by count or hash

2 releases

0.1.1 Nov 25, 2022
0.1.0 Nov 25, 2022

#8 in #partitioning

Download history 87/week @ 2024-09-17 78/week @ 2024-09-24 74/week @ 2024-10-01 60/week @ 2024-10-08 66/week @ 2024-10-15 51/week @ 2024-10-22 42/week @ 2024-10-29 50/week @ 2024-11-05 46/week @ 2024-11-12 130/week @ 2024-11-19 92/week @ 2024-11-26 168/week @ 2024-12-03 172/week @ 2024-12-10 102/week @ 2024-12-17 67/week @ 2024-12-24 34/week @ 2024-12-31

394 downloads per month
Used in 2 crates

MIT/Apache

10KB
232 lines

Introduction

This Rust library allows partitioning a list of tasks to multiple workers, by count or by hash. It was cut out from the awesome nextest.

License

task_partitioner is licensed under the terms of the Apache License (Version 2.0), or MIT.


lib.rs:

Support for partitioning task runs across several machines.

At the moment this only supports simple hash-based and count-based sharding. In the future it could potentially be made smarter: e.g. using data to pick different sets of binaries and tests to run, with an aim to minimize total build and test times.

Dependencies

~0.4–1MB
~20K SLoC