LLM

Note: This crate name previously belonged to another project. The current implementation represents a new and different library. The previous crate is now archived and will not receive any updates. ref: https://github.com/rustformers/llm

LLM is a Rust library that lets you use multiple LLM backends in a single project: OpenAI, Anthropic (Claude), Ollama, DeepSeek, xAI, Phind, Groq and Google. With a unified API and builder style - similar to the Stripe experience - you can easily create chat or text completion requests without multiplying structures and crates.

Key Features

• Multi-backend: Manage OpenAI, Anthropic, Ollama, DeepSeek, xAI, Phind, Groq and Google through a single entry point.
• Multi-step chains: Create multi-step chains with different backends at each step.
• Templates: Use templates to create complex prompts with variables.
• Builder pattern: Configure your LLM (model, temperature, max_tokens, timeouts...) with a few simple calls.
• Chat & Completions: Two unified traits (ChatProvider and CompletionProvider) to cover most use cases.
• Extensible: Easily add new backends.
• Rust-friendly: Designed with clear traits, unified error handling, and conditional compilation via features.
• Validation: Add validation to your requests to ensure the output is what you expect.
• Evaluation: Add evaluation to your requests to score the output of LLMs.
• Function calling: Add function calling to your requests to use tools in your LLMs.
• REST API: Serve any LLM backend as a REST API with openai standard format.

Use any LLM backend on your project

Simply add LLM to your Cargo.toml:

[dependencies]
llm = { version = "1.0.4", features = ["openai", "anthropic", "ollama", "deepseek", "xai", "phind", "google", "groq"] }

Serving any LLM backend as a REST API

• Use standard messages format
• Use step chains to chain multiple LLM backends together
• Expose the chain through a REST API with openai standard format

[dependencies]
llm = { version = "1.0.4", features = ["openai", "anthropic", "ollama", "deepseek", "xai", "phind", "google", "groq", "api"] }

More details in the api_example

More examples

Usage

Here's a basic example using OpenAI for chat completion. See the examples directory for other backends (Anthropic, Ollama, DeepSeek, xAI, Google, Phind), embedding capabilities, and more advanced use cases.