#operator #onnx-runtime #abstraction #interfacing #framework #running #api

ort_custom_op

A library for writing custom operators for the onnxruntime in Rust

9 releases (5 breaking)

0.6.0 Jan 16, 2024
0.5.1 Dec 4, 2023
0.5.0 Oct 6, 2023
0.4.1 Sep 14, 2023
0.1.0 Mar 14, 2023

#497 in Rust patterns

Download history 487/week @ 2024-01-11 1/week @ 2024-01-18 200/week @ 2024-01-25 268/week @ 2024-02-01 63/week @ 2024-02-08 48/week @ 2024-02-15 46/week @ 2024-02-22 32/week @ 2024-02-29 100/week @ 2024-03-07 43/week @ 2024-03-14 59/week @ 2024-03-28 34/week @ 2024-04-04 88/week @ 2024-04-11 168/week @ 2024-04-18

349 downloads per month

BSD-3-Clause

250KB
7K SLoC

Proof of concept for writing custom operators for onnxruntime

Custom operators can be made available to the onnxruntime by creating a shared library with a standardized API. Directly interfacing with the API and types exposed by onnxruntime is quite cumbersome and error prone, though. This project provides abstractions that make this interfacing much easier and safe.

Each custom operator is an individual type which implements the CustomOp trait. Types which implement that trait can be build into static objects which are in turn exposed to the onnxruntime.

The example crate demonstrates how to implement various custom operators. These operators are loaded and used in the Python test cases in tests/python. Building and running these tests requires cargo (i.e. the standard rust tool chain), onnxruntime and pytest.

Execute the following at the root of this repository to build the shared library and to run the python-defined tests:

cargo b && pytest tests/python -s

Dependencies

~1.5MB
~27K SLoC