#async-channel #channel #async #reduce #overhead #values #reading

batch-channel

async channel that reduces overhead by reading and writing many values at once

15 releases

new 0.4.2 Oct 31, 2024
0.4.1 Jun 30, 2024
0.3.7 Oct 11, 2023
0.3.6 Sep 18, 2023
0.1.0 Aug 3, 2023

#617 in Asynchronous

39 downloads per month
Used in photohash

MIT license

38KB
737 lines

async MPMC channel that reduces overhead by reading and writing many values at once.

Sometimes large volumes of small values are farmed out to workers through a channel. Consider directory traversal: each readdir() call produces a batch of directory entries. Batching can help the consumption side too. Consider querying SQLite with received values -- larger batches reduce the amortized cost of acquiring SQLite's lock and bulk queries can be issued.

One can imagine natural, currently unimplemented, extensions to this crate:

  • Channels with priority
  • impls for futures::sink::Sink and futures::stream::Stream

Ask if they would be helpful.

Dependencies

~0.4–25MB
~351K SLoC