Bindings to the llama.cpp library.

As llama.cpp is a very fast moving target, this crate does not attempt to create a stable API with all the rust idioms. Instead it provided safe wrappers around nearly direct bindings to llama.cpp. This makes it easier to keep up with the changes in llama.cpp, but does mean that the API is not as nice as it could be.

Examples

• simple

Feature Flags

• cuda enables CUDA gpu support.
• sampler adds the context::sample::sampler struct for a more rusty way of sampling.

bitnet-cpp

Almost safe wrapper for BitNet-cpp based on llama-cpp

Thanks

Heavily inspired by llama-cpp-rs

License

MIT

Copyright

© 2024, Eugene Hauptmann