Lib.rs

›
#llama #cpp #sampler #bindings #low #api #cuda

sys llama-cpp-sys-4

Low Level Bindings to llama.cpp

Owned by Eugene Hauptmann.

9 releases

new 0.1.91 Jan 9, 2025
0.1.90 Dec 30, 2024
0.1.87 Nov 26, 2024
0.1.83 Oct 27, 2024

#11 in #sampler

Download history 130/week @ 2024-10-26 203/week @ 2024-11-02 71/week @ 2024-11-09 7/week @ 2024-11-16 256/week @ 2024-11-23 20/week @ 2024-11-30 132/week @ 2024-12-07 67/week @ 2024-12-14 115/week @ 2024-12-21 153/week @ 2024-12-28 82/week @ 2025-01-04

420 downloads per month
Used in llama-cpp-4

MIT/Apache

6.5MB
130K SLoC

C++ 79K SLoC // 0.1% comments C 29K SLoC // 0.1% comments CUDA 8K SLoC // 0.0% comments Metal Shading Language 5K SLoC // 0.0% comments Objective-C 4K SLoC // 0.0% comments Python 3.5K SLoC // 0.1% comments Rust 494 SLoC // 0.3% comments

llama-cpp-sys

Raw bindings to llama.cpp with cuda support, including new Sampler API from llama-cpp.

See llama-cpp-4 for a safe API.

OpenMPI support

brew install open-mpi
cargo build -F mpi

lib.rs:

See llama-cpp-4 for a documented and safe API.

Dependencies