#cardinality #count #unique

line_cardinality

High performance line cardinality counts and estimates

4 stable releases

2.0.0 Sep 16, 2024
1.0.2 Aug 11, 2024

#1505 in Algorithms

31 downloads per month
Used in cuniq

GPL-3.0-or-later

44KB
771 lines

line_cardinality

A library that provides high performance line cardinality counts and estimates, including:

  • Hashing with collision detection
  • Hashing without collision detection. Note that collisions are nearly impossible for 64-bit hashes, and this has higher performance due to not having to store lines.
  • [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog

Full API documentation at docs.rs. See PERFORMANCE.md for performance data and technical details on the benchmarking and profile-guided optimization that went into creating line_cardinality.

License

line_cardinality was built primarily for use with the cuniq CLI tool, and is therefore released under the same GPL-3.0-or-later license.

line_cardinality is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

line_cardinality is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

A full list of dependencies is available in Cargo.toml, or a breakdown of dependencies by license can be generated with cargo deny list.

Dependencies

~1.8–2.5MB
~35K SLoC