Cargo Features
[dependencies]
llama-gguf = { version = "0.14.0", default-features = false, features = ["cli", "client", "cpu", "cuda", "vulkan", "vulkan-shaders", "metal", "dx12", "server", "rag", "rag-sqlite", "huggingface", "onnx", "hailo", "distributed"] }- default = cli, client, cpu, huggingface, onnx, server
-
These default features are set whenever
llama-ggufis added withoutdefault-features = false - cli default
-
Enables clap and clap_mangen ^0.2
Required by the binary
- client default
-
Enables reqwest ^0.12
Affects
llama-gguf::client… - cpu default
- cuda
-
Enables cudarc ^0.12
Affects
cuda::dequant_weights,cuda::gpu_only,cuda::CudaBackend,backend::cuda… - vulkan
-
Enables ash and gpu-allocator
Affects
backend::vulkan,vulkan::gpu_only,vulkan::VulkanBackend… - vulkan-shaders
-
Enable build-time shader compilation (requires glslc)
- metal
-
Enables metal ^0.28 and objc
Affects
metal::gpu_only,metal::MetalBackend… - dx12
-
Enables windows ^0.58
Affects
dx12::gpu_only,dx12::Dx12Backend… - server default
-
Enables axum ^0.7, futures, tokio, and tower-http ^0.5
Affects
llama-gguf::engine_batched,llama-gguf::server… - rag
-
Enables deadpool-postgres, glob, pgvector, tokio, tokio-postgres, and url
tokio-postgres:
RAG / pgvector support
Affects
api::ServerConfig.rag_database_url,handlers::RagState,handlers::retrieve,handlers::retrieve_and_generate,handlers::ingest,handlers::list_knowledge_bases,handlers::get_knowledge_base,handlers::delete_knowledge_base… - rag-sqlite
-
Enables glob and rusqlite ^0.31
- huggingface default
-
Enables directories ^5.0, indicatif ^0.17, and reqwest ^0.12
Affects
llama-gguf::huggingface… - onnx default
-
Enables prost ^0.13
ONNX protobuf support
Affects
llama-gguf::onnx… - hailo
-
Enables directories ^5.0 and hailort-sys
Affects
hailo::gpu_only,backend::hailo,engine::EngineConfig.hailo_config… - distributed
-
Enables futures, prost ^0.13, tokio, and tonic ^0.12
tonic:
Distributed inference (gRPC)
Affects
llama-gguf::distributed…