3 unstable releases

0.2.1 Feb 28, 2024
0.2.0 Feb 18, 2024
0.1.0 Dec 16, 2023

#902 in Machine learning

Download history 151/week @ 2024-02-19 212/week @ 2024-02-26 24/week @ 2024-03-04 31/week @ 2024-03-11 10/week @ 2024-03-18 15/week @ 2024-03-25 88/week @ 2024-04-01 2/week @ 2024-04-08 10/week @ 2024-04-22 2/week @ 2024-05-13 20/week @ 2024-05-20 27/week @ 2024-05-27 23/week @ 2024-06-03

72 downloads per month
Used in 7 crates (6 directly)

MIT/Apache

225KB
6K SLoC

Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

Usage (with the RPhi implementation crate)

use rphi::prelude::*;

#[tokio::main]
async fn main() {
    let mut model = Phi::default();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Dependencies

~30–49MB
~821K SLoC