22 unstable releases (7 breaking)

0.8.3 Feb 15, 2025
0.8.1 Dec 7, 2024
0.8.0 Nov 12, 2024
0.6.0 Jun 29, 2024
0.3.1 Nov 12, 2023

#146 in Machine learning

Download history 9483/week @ 2024-11-20 10435/week @ 2024-11-27 10702/week @ 2024-12-04 10734/week @ 2024-12-11 10150/week @ 2024-12-18 7941/week @ 2024-12-25 8919/week @ 2025-01-01 11533/week @ 2025-01-08 12002/week @ 2025-01-15 10494/week @ 2025-01-22 11229/week @ 2025-01-29 11618/week @ 2025-02-05 11492/week @ 2025-02-12 13170/week @ 2025-02-19 11781/week @ 2025-02-26 12607/week @ 2025-03-05

50,880 downloads per month
Used in 126 crates (79 directly)

MIT/Apache

1.5MB
31K SLoC

Rust 27K SLoC // 0.0% comments Metal Shading Language 2.5K SLoC // 0.0% comments CUDA 1.5K SLoC // 0.0% comments

Contains (Zip file, 2KB) tests/fortran_tensor_3d.pth, (Zip file, 2KB) tests/test.pt, (Zip file, 2KB) tests/test_with_key.pt

candle

Minimalist ML framework for Rust


lib.rs:

ML framework for Rust

use candle_core::{Tensor, DType, Device};

let a = Tensor::arange(0f32, 6f32, &Device::Cpu)?.reshape((2, 3))?;
let b = Tensor::arange(0f32, 12f32, &Device::Cpu)?.reshape((3, 4))?;
let c = a.matmul(&b)?;

Features

  • Simple syntax (looks and feels like PyTorch)
  • CPU and Cuda backends (and M1 support)
  • Enable serverless (CPU) small and fast deployments
  • Model training
  • Distributed computing (NCCL).
  • Models out of the box (Llama, Whisper, Falcon, ...)

FAQ

  • Why Candle?

Candle stems from the need to reduce binary size in order to enable serverless possible by making the whole engine smaller than PyTorch very large library volume

And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.

Rust is cool, and a lot of the HF ecosystem already has Rust crates safetensors and tokenizers

Other Crates

Candle consists of a number of crates. This crate holds core the common data structures but you may wish to look at the docs for the other crates which can be found here:

Dependencies

~7–20MB
~354K SLoC