#load #data-loader #channel #task #requests #request #assignment

yanked channel-loader

A channel based solution designed for efficient data loading

0.1.0-beta.12 Jun 23, 2021
0.1.0-beta.11 Jun 22, 2021
0.1.0-beta.4 May 23, 2021

#11 in #data-loader

28 downloads per month

MIT license

24KB
524 lines

Channel Loader

License Cargo Documentation

A work-stealing data loader designed around optimal load batching and connection utilization. Rather than using yields to collect batches of loads as would other data loaders, loads are enqueued to thread local dequeus and task handlers are spawned upfront. As field resolvers within the same request will already be scheduled, subsequent loads within the same request will be scheduled prior to the load task handler taking a task assignment and upon connection acquisition, all load requests will have been captured and loads by separate requests can be opportunistically batched together without introducing timeout overhead.

Dependencies

~4–22MB
~330K SLoC