8 releases

0.3.4 Oct 10, 2024
0.3.3 Aug 21, 2024
0.2.2 Feb 28, 2024
0.1.0 Dec 16, 2023

#907 in Machine learning

Download history 2/week @ 2024-07-04 6/week @ 2024-07-11 6/week @ 2024-07-18 6/week @ 2024-07-25 8/week @ 2024-08-01 194/week @ 2024-08-08 181/week @ 2024-08-15 71/week @ 2024-08-22 19/week @ 2024-08-29 7/week @ 2024-09-05 113/week @ 2024-09-12 188/week @ 2024-09-19 155/week @ 2024-09-26 78/week @ 2024-10-03 205/week @ 2024-10-10 77/week @ 2024-10-17

546 downloads per month
Used in 4 crates (via kalosm-language)

MIT/Apache

395KB
9K SLoC

RLlama

RLlama is a Rust implementation of the quantized Llama 7B language model.

Llama 7B is a very small but performant language model that can be easily run on your local machine.

This library uses Candle to run Llama.

Usage

use kalosm_llama::prelude::*;

#[tokio::main]
async fn main() {
    let mut model = Llama::new().await.unwrap();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Dependencies

~34–55MB
~1M SLoC