Lib.rs

› Science › Machine learning › tract
#linear-algebra #tensorflow #neural-network #onnx #inference #tiny #self-contained

tract-linalg

Tiny, no-nonsense, self contained, TensorFlow and ONNX inference

by Mathieu Poumeyrol and 67 contributors

135 releases

0.21.7 Sep 23, 2024
0.21.6 Jul 24, 2024
0.21.5 May 11, 2024
0.21.2 Mar 29, 2024
0.2.9 Mar 28, 2019

#450 in Machine learning

Download history 5420/week @ 2024-08-01 5169/week @ 2024-08-08 3195/week @ 2024-08-15 6660/week @ 2024-08-22 4762/week @ 2024-08-29 6568/week @ 2024-09-05 6126/week @ 2024-09-12 7214/week @ 2024-09-19 6879/week @ 2024-09-26 7560/week @ 2024-10-03 8257/week @ 2024-10-10 8322/week @ 2024-10-17 10280/week @ 2024-10-24 4678/week @ 2024-10-31 2886/week @ 2024-11-07 2910/week @ 2024-11-14

21,802 downloads per month
Used in 38 crates (2 directly)

MIT/Apache

1MB
23K SLoC

Rust 14K SLoC // 0.0% comments Templ 9K SLoC // 0.1% comments GNU Style Assembly 13 SLoC // 0.3% comments

tract-linalg

linalg stands for "linear algebra". This is a misnamer. This crates contains low-level, architecture dependant optimisations used by tract-core.

Functions

  • MatMatMul: Extended matrix*matrix product:
    • inspired by Gotoblass and BLIS micro kernel approach
    • extended for convolution friendly addressing (fused img2col)
    • fused output pipeline (min, max, and a few more simple, fast ops)
    • f32*f32 -> f32 (à la sgemm)
    • i8*i8 -> i32 accumulator -> i32 storage
    • i8*i8 -> i32 accumulator -> i8 (with channel zeropoint and scale, and re-quantization pipeline)
  • f32 sigmoid and f32 tanh: at f32 precision, by a rationale function (no exponentiation)
  • byte-to-byte lookup table

Implementations

generic fallback armv6, vfp armv7 neon armv8 simd x64 FMA
MatMatMul f32 4x4 8x4 8x8 16x6
MatMatMul i8->i8 8x4 8x8
MatMatMul i8->i32 8x8
sigmoid f32 4n 4n
tanh f32 4n 4n
byte lookup

Dependencies

~10–18MB
~248K SLoC