Lib.rs

›
#llama #cpp #sampler #bindings #low #api #cuda

sys llama-cpp-sys-4

Low Level Bindings to llama.cpp

Owned by Eugene Hauptmann.

10 releases

0.1.92 Jan 21, 2025
0.1.91 Jan 9, 2025
0.1.90 Dec 30, 2024
0.1.87 Nov 26, 2024
0.1.83 Oct 27, 2024

#11 in #sampler

Download history 21/week @ 2024-10-21 118/week @ 2024-10-28 251/week @ 2024-11-04 15/week @ 2024-11-11 144/week @ 2024-11-18 129/week @ 2024-11-25 91/week @ 2024-12-02 60/week @ 2024-12-09 149/week @ 2024-12-16 28/week @ 2024-12-23 163/week @ 2024-12-30 160/week @ 2025-01-06 35/week @ 2025-01-13 111/week @ 2025-01-20

473 downloads per month
Used in llama-cpp-4

MIT/Apache

6.5MB
132K SLoC

C++ 80K SLoC // 0.1% comments C 30K SLoC // 0.1% comments CUDA 9K SLoC // 0.0% comments Metal Shading Language 5K SLoC // 0.0% comments Objective-C 4K SLoC // 0.0% comments Python 4K SLoC // 0.1% comments Rust 496 SLoC // 0.3% comments

llama-cpp-sys

Raw bindings to llama.cpp with cuda support, including new Sampler API from llama-cpp.

See llama-cpp-4 for a safe API.

OpenMPI support

brew install open-mpi
cargo build -F mpi

lib.rs:

See llama-cpp-4 for a documented and safe API.

Dependencies