openllm

SDK for OpenAI compatible APIs.

Usage

Add openllm by using cargo add openllm.

Features

• Embedding API
• Transcription & Translation API
• Speech API
• Chat Completion API with tools
• Chat Completion API streaming
• Chat Completion API with image input
• Create Image API
• Create Image Edit API
• Create Image Variant API

As assistant API is still in Beta and is super slow, so we don't have plan to support it (and relevant file APIs) for now.

Examples

Here are some examples of how to use the SDK:

let sdk = LlmSdk::new_with_base_url("https://api.deepseek.com", "your-api-key");

// chat completion
let messages = vec![
    ChatCompletionMessage::new_system("I can answer any question you ask me.", ""),
    ChatCompletionMessage::new_user("What is human life expectancy in the world?", "user1"),
];
let req = ChatCompletionRequest::new(openllm::ChatCompleteModel::DeepSeekChat, messages);
let res = sdk.chat_completion(req).await?;

// stream
sdk.chat_stream(req, |msg| {
    println!("Received message: {:?}", msg.choices);
})
.await?;

For more usage, please check the test cases.