lmonade

ALPHA SOFTWARE: This is v0.1.0-alpha.1 - an early development release. Basic inference is not yet fully functional. This release is primarily for developers interested in the architecture and contributing to development.

High-performance LLM inference engine built in Rust.

Installation

[dependencies]
lmonade = "0.1.0-alpha.1"

For full functionality including the HTTP server:

[dependencies]
lmonade = { version = "0.1.0-alpha.1", features = ["full"] }

Quick Start

use lmonade::prelude::*;
use lmonade::runtime::ModelHub;

#[tokio::main]
async fn main() -> Result<()> {
    // Create a model hub
    let hub = ModelHub::new(Default::default()).await?;
    
    // Load and use models...
    Ok(())
}

Features

• models - Model architectures and components (default)
• runtime - Actor-based runtime (default)
• server - HTTP API server
• full - All features including server and tokio

Sub-crates

This is a meta-crate that re-exports:

• lmonade-models - Model architectures
• lmonade-runtime - Inference runtime
• lmonade-server - HTTP server

License

GPL-3.0-or-later