3 unstable releases
0.3.0 | Jul 26, 2024 |
---|---|
0.1.1 | Jul 23, 2024 |
0.1.0 | Jul 21, 2024 |
#1304 in Web programming
27KB
600 lines
ollama-rest.rs
Asynchronous Rust bindings of Ollama REST API, using reqwest, tokio, serde, and chrono.
Install
cargo add ollama-rest@0.3
Features
name | status |
---|---|
Completion | Working ✅ |
Embedding | Working ✅ |
Model creation | Working ✅ |
Model deletion | Working ✅ |
Model pulling | Working ✅ |
Model copying | Working ✅ |
Local models | Working ✅ |
Running models | Working ✅ |
Model pushing | Experimental 🧪 |
Tools | Experimental 🧪 |
At a glance
See source of this example.
use std::io::Write;
use ollama_rest::{models::generate::{GenerationRequest, GenerationResponse}, Ollama};
use serde_json::json;
// By default checking Ollama at 127.0.0.1:11434
let ollama = Ollama::default();
let request = serde_json::from_value::<GenerationRequest>(json!({
"model": "llama3",
"prompt": "Why is the sky blue?",
})).unwrap();
let mut stream = ollama.generate_streamed(&request).await.unwrap();
while let Some(Ok(res)) = stream.next().await {
if !res.done {
print!("{}", res.response);
// Flush stdout for each word to allow realtime output
std::io::stdout().flush().unwrap();
}
}
println!();
Or, make your own chatbot interface! See this example (CLI) and this example (REST API).
Dependencies
~7–18MB
~256K SLoC