4 releases
0.2.2 | Feb 28, 2024 |
---|---|
0.2.1 | Feb 28, 2024 |
0.2.0 | Feb 18, 2024 |
0.1.0 | Dec 16, 2023 |
#863 in Machine learning
Used in 2 crates
290KB
7.5K
SLoC
RLlama
RLlama is a Rust implementation of the quantized Llama 7B language model.
Llama 7B is a very small but performant language model that can be easily run on your local machine.
This library uses Candle to run Llama.
Usage
use kalosm_llama::prelude::*;
#[tokio::main]
async fn main() {
let mut model = Llama::default();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await?;
print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}
Dependencies
~31–49MB
~1M SLoC