Lib.rs

› Science › Machine learning › tract
#linear-algebra #tensorflow #neural-network #onnx #low-level #inference #linalg

tract-linalg

Tiny, no-nonsense, self contained, TensorFlow and ONNX inference

by Mathieu Poumeyrol and 64 contributors

134 releases

new 0.21.6 Jul 24, 2024
0.21.4 Apr 23, 2024
0.21.2 Mar 29, 2024
0.20.22 Nov 28, 2023
0.2.9 Mar 28, 2019

#320 in Machine learning

Download history 7638/week @ 2024-04-05 6949/week @ 2024-04-12 9719/week @ 2024-04-19 8239/week @ 2024-04-26 4092/week @ 2024-05-03 5975/week @ 2024-05-10 4287/week @ 2024-05-17 6130/week @ 2024-05-24 6472/week @ 2024-05-31 8948/week @ 2024-06-07 6489/week @ 2024-06-14 8905/week @ 2024-06-21 5067/week @ 2024-06-28 9147/week @ 2024-07-05 5524/week @ 2024-07-12 8539/week @ 2024-07-19

29,505 downloads per month
Used in 39 crates (2 directly)

MIT/Apache

615KB
14K SLoC

tract-linalg

linalg stands for "linear algebra". This is a misnamer. This crates contains low-level, architecture dependant optimisations used by tract-core.

Functions

  • MatMatMul: Extended matrix*matrix product:
    • inspired by Gotoblass and BLIS micro kernel approach
    • extended for convolution friendly addressing (fused img2col)
    • fused output pipeline (min, max, and a few more simple, fast ops)
    • f32*f32 -> f32 (à la sgemm)
    • i8*i8 -> i32 accumulator -> i32 storage
    • i8*i8 -> i32 accumulator -> i8 (with channel zeropoint and scale, and re-quantization pipeline)
  • f32 sigmoid and f32 tanh: at f32 precision, by a rationale function (no exponentiation)
  • byte-to-byte lookup table

Implementations

generic fallback armv6, vfp armv7 neon armv8 simd x64 FMA
MatMatMul f32 4x4 8x4 8x8 16x6
MatMatMul i8->i8 8x4 8x8
MatMatMul i8->i32 8x8
sigmoid f32 4n 4n
tanh f32 4n 4n
byte lookup

Dependencies

~9–13MB
~230K SLoC