2 stable releases

5.0.1	Jan 28, 2026

#294 in Asynchronous

MIT license

1.5MB
38K SLoC

🦜️🔗LangChain AI Rust

⚡ Building applications with LLMs through composability, with Rust! ⚡

🤔 What is this?

This is the Rust language implementation of LangChain, providing a powerful and type-safe way to build LLM applications in Rust.

✨ Key Features

🚀 Multiple LLM Providers: Support for OpenAI, Azure OpenAI, Anthropic Claude, MistralAI, Google Gemini, AWS Bedrock, HuggingFace, Alibaba Qwen, DeepSeek, and Ollama
🔗 Chains: LLM chains, conversational chains, sequential chains, Q&A chains, SQL chains, and more
🤖 Agents: Chat agents with tools, multi-agent systems (router, subagents, skills, handoffs)
📚 RAG: Agentic RAG, Hybrid RAG, and two-step RAG implementations
🧠 Memory: Simple memory, conversational memory, and long-term memory with metadata
🛠️ Tools: Search tools, command line, Wolfram Alpha, text-to-speech, and more
📄 Document Loaders: PDF, HTML, CSV, Git commits, source code, and more
🗄️ Vector Stores: PostgreSQL (pgvector), Qdrant, SQLite (VSS/Vec), SurrealDB, OpenSearch, In-Memory, Chroma, FAISS (hnsw_rs), MongoDB Atlas, Pinecone, Weaviate
🎯 Embeddings: OpenAI, Azure OpenAI, Ollama, FastEmbed, MistralAI
🔧 Middleware: Logging, PII detection, content filtering, rate limiting, retry, and custom middleware
🎨 Structured Output: JSON schema validation and structured response generation
⚙️ Runtime Context: Dynamic prompts, typed context, and runtime-aware middleware
📊 LangGraph: State graphs, streaming, persistence (SQLite/memory), interrupts, subgraphs, and time-travel debugging
🤖 Deep Agent: Planning (write_todos), filesystem tools (ls, read_file, write_file, edit_file), skills, long-term memory, and human-in-the-loop

📦 Installation

This library heavily relies on serde_json for its operation.

Step 1: Add `serde_json`

First, ensure serde_json is added to your Rust project.

cargo add serde_json

Step 2: Add `langchain-ai-rust`

Then, you can add langchain-ai-rust to your Rust project.

Simple install

cargo add langchain-ai-rust

With Vector Stores

PostgreSQL (pgvector)

cargo add langchain-ai-rust --features postgres

Qdrant

cargo add langchain-ai-rust --features qdrant

SQLite (VSS)

Download additional sqlite_vss libraries from https://github.com/asg017/sqlite-vss

cargo add langchain-ai-rust --features sqlite-vss

SQLite (Vec)

Download additional sqlite_vec libraries from https://github.com/asg017/sqlite-vec

cargo add langchain-ai-rust --features sqlite-vec

SurrealDB

cargo add langchain-ai-rust --features surrealdb

OpenSearch

cargo add langchain-ai-rust --features opensearch

In-Memory

cargo add langchain-ai-rust --features in-memory

Chroma

cargo add langchain-ai-rust --features chroma

FAISS (hnsw_rs)

cargo add langchain-ai-rust --features faiss

MongoDB Atlas Vector Search

cargo add langchain-ai-rust --features mongodb

Pinecone

cargo add langchain-ai-rust --features pinecone

Weaviate

cargo add langchain-ai-rust --features weaviate

With LLM Providers

Ollama

cargo add langchain-ai-rust --features ollama

MistralAI

cargo add langchain-ai-rust --features mistralai

Google Gemini

cargo add langchain-ai-rust --features gemini

AWS Bedrock

cargo add langchain-ai-rust --features bedrock

With Document Loaders

PDF (pdf-extract)

cargo add langchain-ai-rust --features pdf-extract

PDF (lopdf)

cargo add langchain-ai-rust --features lopdf

HTML to Markdown

cargo add langchain-ai-rust --features html-to-markdown

With Code Parsing

Tree-sitter (for source code parsing, requires 0.26+)

cargo add langchain-ai-rust --features tree-sitter

With FastEmbed (Local Embeddings)

cargo add langchain-ai-rust --features fastembed

🚀 Quick Start

Simple LLM Invocation

use langchain_ai_rust::llm::openai::{OpenAI, OpenAIModel};

#[tokio::main]
async fn main() {
    let llm = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());
    let response = llm.invoke("What is Rust?").await.unwrap();
    println!("{}", response);
}

Using init_chat_model (Recommended)

The init_chat_model function provides a unified interface to initialize any supported LLM:

use langchain_ai_rust::language_models::init_chat_model;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize any supported model
    let model = init_chat_model("gpt-4o-mini", None, None, None, None, None, None, None).await?;

    let response = model.invoke("Hello, world!").await?;
    println!("{}", response);

    Ok(())
}

Supported model formats:

gpt-4o-mini, gpt-4o, gpt-4-turbo (OpenAI)
claude-3-5-sonnet-20241022 (Anthropic)
mistralai/mistral-large-latest (MistralAI)
gemini-1.5-pro (Google Gemini)
anthropic.claude-3-5-sonnet-20241022-v2:0 (AWS Bedrock)
meta-llama/Llama-3.1-8B-Instruct (HuggingFace)
qwen-plus (Alibaba Qwen)
deepseek-chat (DeepSeek)
llama3 (Ollama)

Conversational Chain

use langchain_ai_rust::{
    chain::{Chain, LLMChainBuilder},
    fmt_message, fmt_placeholder, fmt_template,
    llm::openai::{OpenAI, OpenAIModel},
    message_formatter,
    prompt::HumanMessagePromptTemplate,
    prompt_args,
    schemas::messages::Message,
    template_fstring,
};

#[tokio::main]
async fn main() {
    let open_ai = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());

    let prompt = message_formatter![
        fmt_message!(Message::new_system_message(
            "You are a helpful assistant."
        )),
        fmt_placeholder!("history"),
        fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
            "{input}", "input"
        ))),
    ];

    let chain = LLMChainBuilder::new()
        .prompt(prompt)
        .llm(open_ai)
        .build()
        .unwrap();

    match chain
        .invoke(prompt_args! {
            "input" => "What is Rust?",
            "history" => vec![
                Message::new_human_message("Hello"),
                Message::new_ai_message("Hi there!"),
            ],
        })
        .await
    {
        Ok(result) => println!("Result: {:?}", result),
        Err(e) => panic!("Error: {:?}", e),
    }
}

Creating an Agent with Tools

use std::sync::Arc;
use langchain_ai_rust::{
    agent::create_agent,
    schemas::messages::Message,
    tools::CommandExecutor,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let command_executor = Arc::new(CommandExecutor::default());

    let agent = create_agent(
        "gpt-4o-mini",
        &[command_executor],
        Some("You are a helpful assistant that can execute commands."),
        None,
    )?;

    let result = agent
        .invoke_messages(vec![Message::new_human_message(
            "What files are in the current directory?",
        )])
        .await?;

    println!("{}", result);
    Ok(())
}

LangGraph (Hello World)

Build a state graph with MessagesState, add a node with function_node, connect START to the node and the node to END, then compile and invoke:

use langchain_ai_rust::langgraph::{function_node, MessagesState, StateGraph, END, START};
use langchain_ai_rust::schemas::messages::Message;

let mock_llm = function_node("mock_llm", |_state: &MessagesState| async move {
    use std::collections::HashMap;
    let mut update = HashMap::new();
    update.insert(
        "messages".to_string(),
        serde_json::to_value(vec![Message::new_ai_message("hello world")])?,
    );
    Ok(update)
});

let mut graph = StateGraph::<MessagesState>::new();
graph.add_node("mock_llm", mock_llm)?;
graph.add_edge(START, "mock_llm");
graph.add_edge("mock_llm", END);

let compiled = graph.compile()?;
let initial_state = MessagesState::with_messages(vec![Message::new_human_message("hi!")]);
let final_state = compiled.invoke(initial_state).await?;

See LangGraph Hello World and LangGraph Streaming for more.

Deep Agent (Basic)

Use create_deep_agent with planning and filesystem enabled; the agent gets a workspace and built-in tools (write_todos, ls, read_file, write_file, edit_file):

use langchain_ai_rust::{
    agent::{create_deep_agent, DeepAgentConfig},
    chain::Chain,
    prompt_args,
    schemas::messages::Message,
};

let workspace = std::env::temp_dir().join("my_agent_workspace");
std::fs::create_dir_all(&workspace)?;

let config = DeepAgentConfig::new()
    .with_planning(true)
    .with_filesystem(true)
    .with_workspace_root(workspace);

let agent = create_deep_agent(
    "gpt-4o-mini",
    &[],
    Some("You are a helpful assistant with planning and file tools."),
    config,
)?;

let result = agent
    .invoke(prompt_args! {
        "messages" => vec![Message::new_human_message("List files in the workspace.")]
    })
    .await?;

See Deep Agent Basic and Deep Agent Customization for more.

📚 Current Features

LLMs

Embeddings

Vector Stores

Chains

Agents

LangGraph

Deep Agent

Text Splitters

Text Structure-Based

RecursiveCharacterTextSplitter - Recommended default, splits recursively by separators
CharacterTextSplitter - Simple character-based splitting with single separator
PlainTextSplitter - Basic text splitting
TokenSplitter - Token-based splitting (Tiktoken)

Document Structure-Based

MarkdownSplitter - Split Markdown by structure
HTMLSplitter - Split HTML by tags
JsonSplitter - Split JSON by objects/arrays
CodeSplitter - Split code by syntax tree (tree-sitter 0.26+, requires tree-sitter feature)

RAG (Retrieval-Augmented Generation)

Agentic RAG - Agent decides when to retrieve
Hybrid RAG - Combines multiple retrieval strategies
Two-Step RAG - Two-stage retrieval process

Retrievers

External Index Retrievers

Wikipedia Retriever - Retrieve Wikipedia articles
Arxiv Retriever - Retrieve academic papers from arXiv
Tavily Search API Retriever - Real-time web search

Algorithm-Based Retrievers

BM25 Retriever - BM25 algorithm for text retrieval
TF-IDF Retriever - TF-IDF based retrieval
SVM Retriever - Support Vector Machine based retrieval

Rerankers

Cohere Reranker - Rerank using Cohere API
FlashRank Reranker - Local ONNX model reranking
Contextual AI Reranker - Contextual AI API reranking

Hybrid Retrievers

Merger Retriever - Combine multiple retrievers
Ensemble Retriever - Voting mechanism from multiple retrievers

Query Enhancement Retrievers

RePhrase Query Retriever - LLM-based query rephrasing
Multi Query Retriever - Generate multiple query variations

Document Compression Retrievers

Embeddings Redundant Filter - Filter redundant documents by similarity

Tools

Serpapi/Google Search
DuckDuckGo Search
Wolfram Alpha
Command Line Executor
Text-to-Speech
Speech-to-Text
Advanced Tools

Middleware

Memory

Runtime & Context

Structured Output

Advanced Features

Document Loaders

Common File Types

PDF (pdf-extract or lopdf)
HTML
HTML to Markdown
CSV
TSV (Tab-Separated Values)
JSON (including JSONL)
Markdown
TOML (with toml feature)
YAML (with yaml feature)
XML (with xml feature)

Office Documents

Excel (.xlsx, .xls) (with excel feature)
Word, PowerPoint, and more via PandocLoader

Web Loaders

WebBaseLoader - Load content from URLs
RecursiveURLLoader - Recursively crawl websites
SitemapLoader - Load all URLs from sitemap.xml (with xml feature)

Cloud Storage

AWS S3 (with aws-s3 feature)

Productivity Tools

GitHub (with github feature)
Git Commits (with git feature)

Other

Source Code (with tree-sitter feature)
Pandoc (various formats: docx, epub, html, ipynb, markdown, etc.)

See the examples directory for complete examples of each feature.

🔧 Configuration

Environment Variables

For OpenAI:

export OPENAI_API_KEY="your-api-key"

For Anthropic:

export ANTHROPIC_API_KEY="your-api-key"

For MistralAI:

export MISTRAL_API_KEY="your-api-key"

For Google Gemini:

export GOOGLE_API_KEY="your-api-key"

For AWS Bedrock:

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION="us-east-1"

📖 Documentation

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.