Cargo Features

[dependencies]
llama_cpp_sys = { version = "0.3.2", default-features = false, features = ["compat", "native", "avx2", "avx512_vnni", "fma", "f16c", "accel", "mpi", "cuda", "cuda_f16", "cuda_dmmv", "cuda_mmq", "clblast", "vulkan"] }
default = compat, native

These default features are set whenever llama_cpp_sys is added without default-features = false somewhere in the dependency tree.

compat default

this feature modifies the symbols exposed by the generated libraries to avoid conflicts

native default = accel, avx, avx2, f16c, fma
avx avx2 native
avx512 avx512_vmbi avx512_vnni
fma native
f16c native

implied when compiled using MSVC with avx2/avx512

accel native

Accelerate framework

mpi
cuda cuda_dmmv? cuda_f16? cuda_mmq?

Enables cudarc ^0.10.0

cuda_f16 = cuda
cuda_dmmv = cuda

use dmmv instead of mmvq CUDA kernels

cuda_mmq = cuda

use mmq kernels instead of cuBLAS

metal blas hipblas clblast
vulkan

Enables ash ^0.37.3