#onnx #pipeline #ort

orp

Lightweight framework for building ONNX runtime pipelines with ort

3 releases

0.9.2 Mar 23, 2025
0.9.1 Jan 30, 2025
0.9.0 Jan 30, 2025

#508 in Machine learning

Download history 225/week @ 2025-01-27 45/week @ 2025-02-03 4/week @ 2025-02-10 8/week @ 2025-02-17 33/week @ 2025-02-24 253/week @ 2025-03-03 296/week @ 2025-03-10 151/week @ 2025-03-17 211/week @ 2025-03-24

914 downloads per month
Used in 3 crates

Apache-2.0

17KB
183 lines

🧩 ORP: a Lightweight Rust Framework for Building ONNX Runtime Pipelines with ORT

💬 Introduction

orp is a lightweight framework designed to simplify the creation and execution of ONNX Runtime Pipelines in Rust. Built on top of the 🦀 ort runtime and the 🔗 composable crate, it provides an simple way to handle data pre- and post-processing, chain multiple ONNX models together, while encouraging code reuse and clarity.

🔨 Sample Use-Cases

⚡️ GPU/NPU Inferences

The execution providers available in ort can be leveraged to perform considerably faster inferences on GPU/NPU hardware.

The first step is to pass the appropriate execution providers in RuntimeParameters. For example:

let rtp = RuntimeParameters::default().with_execution_providers([
    CUDAExecutionProvider::default().build()
]);

The second step is to activate the appropriate features (see related section below), otherwise ir may silently fall-back to CPU. For example:

$ cargo run --features=cuda ...

Please refer to doc/ORT.md for details about execution providers.

📦 Crate Features

This create mirrors the following ort features:

  • To allow for dynamic loading of ONNX-runtime libraries: load-dynamic
  • To allow for activation of execution providers: cuda, tensorrt, directml, coreml, rocm, openvino, onednn, xnnpack, qnn, cann, nnapi, tvm, acl, armnn, migraphx, vitis, and rknpu.

⚙️ Dependencies

  • ort: the ONNX runtime wrapper
  • composable: this crate is used to actually define the pre- and post-processing pipelines by composition or elementary steps, and can in turn be used to combine mutliple pipelines.

Dependencies

~2.5–7.5MB
~54K SLoC