#edit-distance #lzjd #ziv #lempel

bin+lib malwaredb-lzjd

Rust implementation of the LZJD algorithm by Edward Raff -- https://github.com/EdwardRaff/jLZJD

3 releases

0.1.3 May 6, 2024
0.1.2 Sep 17, 2023
0.1.1 Sep 17, 2023
0.1.0 Aug 22, 2023

#683 in Cryptography

Download history 125/week @ 2024-12-14 57/week @ 2024-12-21 81/week @ 2024-12-28 100/week @ 2025-01-04 142/week @ 2025-01-11 76/week @ 2025-01-18 176/week @ 2025-01-25 243/week @ 2025-02-01 127/week @ 2025-02-08 156/week @ 2025-02-15 208/week @ 2025-02-22 92/week @ 2025-03-01 196/week @ 2025-03-08 251/week @ 2025-03-15 45/week @ 2025-03-22 167/week @ 2025-03-29

677 downloads per month
Used in 3 crates (2 directly)

GPL-3.0 license

28KB
567 lines

TestLintCrates.io Version

LZJD

Documentation

Rust implementation of Lempel-Ziv Jaccard Distance (LZJD) algorithm based on jLZJD by Edward Raff.

Main differences:

  • Rust instead of Java
  • Can use any hasher (executable uses CRC32) instead of just Murmur3
  • Does not allocate memory for every unique hash, instead keeps k=1024 smallest
  • Based on Vec<u64> instead of IntSetNoRemove, which is more like HashMap
  • Hash files are considerably smaller if small sequences have been digested

This fork has minor changes:

  • Update to Rust edition 2021.
  • Remove dependencies preventing it from working on non-x86 hardware.
USAGE:
    lzjd [FLAGS] [OPTIONS] <INPUT>...

FLAGS:
    -c, --compare        compare SDBFs in file, or two SDBF files
    -r, --deep           generate SDBFs from directories and files
    -g, --gen-compare    compare all pairs in source data
    -h, --help           Prints help information
    -V, --version        Prints version information

OPTIONS:
    -o, --output <FILE>            send output to files
    -t, --threshold <THRESHOLD>    only show results >= threshold [default: 1]

ARGS:
    <INPUT>...    Sets the input file to use

See also:

Dependencies

~3–10MB
~107K SLoC