Cargo Features
[dependencies]
llama_cpp_sys = { version = "0.3.2", default-features = false, features = ["compat", "native", "avx2", "avx512_vnni", "fma", "f16c", "accel", "mpi", "cuda", "cuda_f16", "cuda_dmmv", "cuda_mmq", "clblast", "vulkan"] }- default = compat, native
-
These default features are set whenever
llama_cpp_sysis added withoutdefault-features = false - compat default
-
this feature modifies the symbols exposed by the generated libraries to avoid conflicts
- native default = accel, avx, avx2, f16c, fma
- avx avx2 native
- avx512 avx512_vmbi avx512_vnni
- fma native
- f16c native
-
implied when compiled using MSVC with avx2/avx512
- accel native
-
Accelerate framework
- mpi
- cuda cuda_dmmv? cuda_f16? cuda_mmq?
-
Enables cudarc ^0.10.0
- cuda_f16 = cuda
- cuda_dmmv = cuda
-
use dmmv instead of mmvq CUDA kernels
- cuda_mmq = cuda
-
use mmq kernels instead of cuBLAS
- metal blas hipblas clblast
- vulkan
-
Enables ash ^0.37.3