Cargo Features
[dependencies]
llama_cpp = { version = "0.3.2", default-features = false, features = ["compat", "native", "avx", "avx2", "avx512", "avx512_vmbi", "avx512_vnni", "fma", "f16c", "accel", "mpi", "cuda", "cuda_f16", "cuda_dmmv", "cuda_mmq", "metal", "blas", "hipblas", "clblast", "vulkan", "sys_verbosity"] }
- default = compat, native
-
These default features are set whenever
llama_cpp
is added without
somewhere in the dependency tree.default-features = false - compat default
-
this feature modifies the symbols exposed by the generated libraries to avoid conflicts
Enables compat of llama_cpp_sys
- native default = accel, avx, avx2, f16c, fma
-
Enables native of llama_cpp_sys
- avx native
-
Enables avx of llama_cpp_sys
- avx2 native
-
Enables avx2 of llama_cpp_sys
- avx512
-
Enables avx512 of llama_cpp_sys
- avx512_vmbi
-
Enables avx512_vmbi of llama_cpp_sys
- avx512_vnni
-
Enables avx512_vnni of llama_cpp_sys
- fma native
-
Enables fma of llama_cpp_sys
- f16c native
-
implied when compiled using MSVC with avx2/avx512
Enables f16c of llama_cpp_sys
- accel native
-
Accelerate framework
Enables accel of llama_cpp_sys
- mpi
-
Enables mpi of llama_cpp_sys
- cuda cuda_dmmv? cuda_f16? cuda_mmq?
-
Enables cuda of llama_cpp_sys
- cuda_f16 = cuda
-
Enables cuda_f16 of llama_cpp_sys
- cuda_dmmv = cuda
-
use dmmv instead of mmvq CUDA kernels
Enables cuda_dmmv of llama_cpp_sys
- cuda_mmq = cuda
-
use mmq kernels instead of cuBLAS
Enables cuda_mmq of llama_cpp_sys
- metal
-
Enables metal of llama_cpp_sys
- blas
-
Enables blas of llama_cpp_sys
- hipblas
-
Enables hipblas of llama_cpp_sys
- clblast
-
Enables clblast of llama_cpp_sys
- vulkan
-
Enables vulkan of llama_cpp_sys
- sys_verbosity
-
increase sys crate tracing log verbosity level