#crc #crc64 #simd #checksum

crc64fast

SIMD accelerated CRC64 calculation

2 releases (1 stable)

1.0.0 May 2, 2020
0.1.0 Nov 20, 2019

#4 in #crc64

Download history 585/week @ 2021-10-05 587/week @ 2021-10-12 501/week @ 2021-10-19 539/week @ 2021-10-26 614/week @ 2021-11-02 704/week @ 2021-11-09 927/week @ 2021-11-16 746/week @ 2021-11-23 928/week @ 2021-11-30 780/week @ 2021-12-07 746/week @ 2021-12-14 683/week @ 2021-12-21 766/week @ 2021-12-28 886/week @ 2022-01-04 779/week @ 2022-01-11 399/week @ 2022-01-18

2,925 downloads per month
Used in 2 crates

MIT/Apache

125KB
4.5K SLoC

crc64fast

Build status Latest Version Documentation

SIMD-accelerated CRC-64-ECMA computation (similar to crc32fast).

Usage

use crc64fast::Digest;

let mut c = Digest::new();
c.write(b"hello ");
c.write(b"world!");
let checksum = c.sum64();
assert_eq!(checksum, 0x8483_c0fa_3260_7d61);

Performance

crc64fast provides two fast implementations, and the most performance one will be chosen based on CPU feature at runtime.

  • a fast, platform-agnostic table-based implementation, processing 16 bytes at a time.
  • a SIMD-carryless-multiplication based implementation on modern processors:
    • using PCLMULQDQ + SSE 4.1 on x86/x86_64
    • using PMULL + NEON on AArch64 (64-bit ARM)
Algorithm Throughput (x86_64) Throughput (aarch64)
crc 1.8.1 0.5 GiB/s 0.3 GiB/s
crc64fast (table) 2.3 GiB/s 1.8 GiB/s
crc64fast (simd) 28.2 GiB/s 20.0 GiB/s

Note: Since Rust has not stabilized SIMD support on AArch64, you need a nightly compiler and enable the pmull feature to use the SIMD-based implementation:

[dependencies]
crc64fast = { version = "1.0", features = ["pmull"] }

TODO

This crate is mainly intended for use in TiKV only. Features beyond AArch64 are unlikely to be implemented.

  • AArch64 support based on PMULL
  • no_std support
  • Fuzz test
  • Custom polynomial

License

crc64fast is dual-licensed under

No runtime deps