2 releases
| 0.1.3 | Dec 17, 2024 |
|---|---|
| 0.1.1 | Dec 17, 2024 |
#571 in HTTP server
24KB
454 lines
HTMX + Llama.cpp Server ❤️
On machine with llama.cpp
.\llama-server -ngl 100 --port 9090 -m <some.gguf> --host 0.0.0.0
When running
cargo run -- --llama http://<llama.cpp_server_IP>:9090
Dependencies
~23–36MB
~651K SLoC