#project

batch-mode-batch-workflow

This crate provides a specialized approach to batch-based langauge model expansions and reconciling partial or incomplete batch states

1 unstable release

Uses new Rust 2024

new 0.1.1 Mar 31, 2025

#791 in #project


Used in 3 crates

MIT license

795KB
13K SLoC

batch-mode-batch-workflow

This crate provides a specialized approach to batch-based GPT expansions and reconciling partial or incomplete batch states. It defines traits and workflows for:

  • Reconciling uncompleted batches from prior runs,
  • Computing new requests for language model expansions,
  • Processing these requests in batches (locally or via remote APIs),
  • Handling responses and results systematically.

Key Traits

  • FinishProcessingUncompletedBatches – Finalize partial data left from incomplete batch processing.
  • ComputeLanguageModelRequests – Identify new items to process and generate requests to a language model API.
  • ProcessBatchRequests – Handle chunked batch requests.
  • LanguageModelBatchWorkflow – Integrates the above traits into a high-level, end-to-end batch-processing workflow.

Getting Started

Add this crate to your Cargo.toml:

[dependencies]
batch-mode-batch-workflow = "*"

Dependencies

~26–43MB
~665K SLoC