Lib.rs

› Science › Machine learning › tract
#linear-algebra #tensorflow #neural-network #onnx #inference #tiny #self-contained

tract-linalg

Tiny, no-nonsense, self contained, TensorFlow and ONNX inference

by Mathieu Poumeyrol and 67 contributors

136 releases

new 0.21.8 Dec 5, 2024
0.21.7 Sep 23, 2024
0.21.6 Jul 24, 2024
0.21.2 Mar 29, 2024
0.2.9 Mar 28, 2019

#604 in Machine learning

Download history 5338/week @ 2024-08-17 5664/week @ 2024-08-24 5489/week @ 2024-08-31 7095/week @ 2024-09-07 5917/week @ 2024-09-14 7538/week @ 2024-09-21 6797/week @ 2024-09-28 7032/week @ 2024-10-05 10625/week @ 2024-10-12 9841/week @ 2024-10-19 5445/week @ 2024-10-26 4380/week @ 2024-11-02 2504/week @ 2024-11-09 3790/week @ 2024-11-16 4844/week @ 2024-11-23 3513/week @ 2024-11-30

15,345 downloads per month
Used in 38 crates (2 directly)

MIT/Apache

1MB
26K SLoC

Rust 17K SLoC // 0.0% comments Templ 9K SLoC // 0.1% comments GNU Style Assembly 13 SLoC // 0.3% comments

tract-linalg

linalg stands for "linear algebra". This is a misnamer. This crates contains low-level, architecture dependant optimisations used by tract-core.

Functions

  • MatMatMul: Extended matrix*matrix product:
    • inspired by Gotoblass and BLIS micro kernel approach
    • extended for convolution friendly addressing (fused img2col)
    • fused output pipeline (min, max, and a few more simple, fast ops)
    • f32*f32 -> f32 (à la sgemm)
    • i8*i8 -> i32 accumulator -> i32 storage
    • i8*i8 -> i32 accumulator -> i8 (with channel zeropoint and scale, and re-quantization pipeline)
  • f32 sigmoid and f32 tanh: at f32 precision, by a rationale function (no exponentiation)
  • byte-to-byte lookup table

Implementations

generic fallback armv6, vfp armv7 neon armv8 simd x64 FMA
MatMatMul f32 4x4 8x4 8x8 16x6
MatMatMul i8->i8 8x4 8x8
MatMatMul i8->i32 8x8
sigmoid f32 4n 4n
tanh f32 4n 4n
byte lookup

Dependencies

~10–17MB
~248K SLoC