12 releases (breaking)

0.9.0 Dec 3, 2021
0.8.0 Nov 30, 2021
0.7.0 Jul 18, 2021
0.6.0 Feb 21, 2021
0.2.0 Feb 18, 2019

#1249 in Algorithms

Download history 116/week @ 2023-11-20 101/week @ 2023-11-27 91/week @ 2023-12-04 117/week @ 2023-12-11 101/week @ 2023-12-18 6/week @ 2023-12-25 25/week @ 2024-01-01 57/week @ 2024-01-08 90/week @ 2024-01-15 261/week @ 2024-01-22 217/week @ 2024-01-29 212/week @ 2024-02-05 352/week @ 2024-02-12 1048/week @ 2024-02-19 264/week @ 2024-02-26 329/week @ 2024-03-04

2,015 downloads per month
Used in 11 crates (4 directly)

Apache-2.0

76KB
1.5K SLoC

Reductive

Training of optimized product quantizers

Training of optimized product quantizers requires a LAPACK implementation. For this reason, training of the Opq and GaussianOpq quantizers is feature-gated by the opq-train feature. This feature must be enabled if you want to use Opq or GaussianOpq:

[dependencies]
reductive = { version = "0.7", features = ["opq-train"] }

This also requires that a crate that links a LAPACK library is added as a dependency, e.g. accelerate-src, intel-mkl-src, openblas-src, or netlib-src.

Running tests

Linux

You can run all tests on Linux, including tests for optimized product quantizers, using the intel-mkl-test feature:

$ cargo test --features intel-mkl-test

macOS

All tests can be run on macOS with the accelerate-test feature:

$ cargo test --features accelerate-test

Multi-threaded OpenBLAS

reductive uses Rayon to parallelize quantizer training. However, multi-threaded OpenBLAS is known to conflict with application threading. Is you use OpenBLAS, ensure that threading is disabled, for instance by setting the number of threads to 1:

$ export OPENBLAS_NUM_THREADS=1

Dependencies

~3–13MB
~203K SLoC