#onnx #pipeline #ort #model

orp

Lightweight framework for building ONNX runtime pipelines with ort

3 releases

0.9.2 Mar 23, 2025
0.9.1 Jan 30, 2025
0.9.0 Jan 30, 2025

#371 in Machine learning

Download history 252/week @ 2025-01-29 20/week @ 2025-02-05 3/week @ 2025-02-12 8/week @ 2025-02-19 49/week @ 2025-02-26 364/week @ 2025-03-05 217/week @ 2025-03-12 185/week @ 2025-03-19 258/week @ 2025-03-26 365/week @ 2025-04-02 88/week @ 2025-04-09

905 downloads per month
Used in 3 crates

Apache-2.0

17KB
183 lines

🧩 ORP: a Lightweight Rust Framework for Building ONNX Runtime Pipelines with ORT

💬 Introduction

orp is a lightweight framework designed to simplify the creation and execution of ONNX Runtime Pipelines in Rust. Built on top of the 🦀 ort runtime and the 🔗 composable crate, it provides an simple way to handle data pre- and post-processing, chain multiple ONNX models together, while encouraging code reuse and clarity.

🔨 Sample Use-Cases

⚡️ GPU/NPU Inferences

The execution providers available in ort can be leveraged to perform considerably faster inferences on GPU/NPU hardware.

The first step is to pass the appropriate execution providers in RuntimeParameters. For example:

let rtp = RuntimeParameters::default().with_execution_providers([
    CUDAExecutionProvider::default().build()
]);

The second step is to activate the appropriate features (see related section below), otherwise ir may silently fall-back to CPU. For example:

$ cargo run --features=cuda ...

Please refer to doc/ORT.md for details about execution providers.

📦 Crate Features

This create mirrors the following ort features:

  • To allow for dynamic loading of ONNX-runtime libraries: load-dynamic
  • To allow for activation of execution providers: cuda, tensorrt, directml, coreml, rocm, openvino, onednn, xnnpack, qnn, cann, nnapi, tvm, acl, armnn, migraphx, vitis, and rknpu.

⚙️ Dependencies

  • ort: the ONNX runtime wrapper
  • composable: this crate is used to actually define the pre- and post-processing pipelines by composition or elementary steps, and can in turn be used to combine mutliple pipelines.

Dependencies

~2.5–7.5MB
~54K SLoC