#llama #cpp #sampler #bindings #low #api #cuda

sys llama-cpp-sys-4

Low Level Bindings to llama.cpp

12 releases

new 0.1.94 Feb 11, 2025
0.1.93 Feb 2, 2025
0.1.92 Jan 21, 2025
0.1.90 Dec 30, 2024
0.1.83 Oct 27, 2024

#12 in #sampler

Download history 39/week @ 2024-10-29 256/week @ 2024-11-05 9/week @ 2024-11-12 147/week @ 2024-11-19 127/week @ 2024-11-26 108/week @ 2024-12-03 45/week @ 2024-12-10 155/week @ 2024-12-17 125/week @ 2024-12-24 66/week @ 2024-12-31 167/week @ 2025-01-07 21/week @ 2025-01-14 111/week @ 2025-01-21 114/week @ 2025-01-28 31/week @ 2025-02-04 125/week @ 2025-02-11

383 downloads per month
Used in llama-cpp-4

MIT/Apache

7MB
134K SLoC

C++ 81K SLoC // 0.1% comments C 30K SLoC // 0.1% comments CUDA 9K SLoC // 0.0% comments Metal Shading Language 5K SLoC // 0.0% comments Objective-C 4K SLoC // 0.0% comments Python 4K SLoC // 0.1% comments Rust 501 SLoC // 0.3% comments

llama-cpp-sys

Raw bindings to llama.cpp with cuda support, including new Sampler API from llama-cpp.

See llama-cpp-4 for a safe API.

OpenMPI support

brew install open-mpi
cargo build -F mpi

lib.rs:

See llama-cpp-4 for a documented and safe API.

Dependencies