3 unstable releases
0.2.0 | Mar 13, 2024 |
---|---|
0.1.1 | Oct 5, 2023 |
0.1.0 | Oct 5, 2023 |
#439 in Machine learning
88 downloads per month
1.5MB
32K
SLoC
Rust bindings to the Triton Inference Server
Triton Rust API
Implementing a backend
use triton_rs::Backend;
struct ExampleBackend;
impl Backend for ExampleBackend {
fn model_instance_execute(
model: triton_rs::Model,
requests: &[triton_rs::Request],
) -> Result<(), triton_rs::Error> {
for request in requests {
// Handle inference request here
todo!();
}
Ok(())
}
}
// Register the backend with Triton
triton_rs::declare_backend!(ExampleBackend);
See example-backend for full example.
Dependencies
~0–1.8MB
~37K SLoC