#image #fuzzy #hash #pdq

pdqhash

Perceptual Hashing, originally designed by Facebook. Identify the same image even after some modifications.

2 releases

0.1.1 Jul 8, 2022
0.1.0 Jul 8, 2022

#23 in #fuzzy

Download history 101/week @ 2022-08-11 132/week @ 2022-08-18 83/week @ 2022-08-25 127/week @ 2022-09-01 98/week @ 2022-09-08 68/week @ 2022-09-15 76/week @ 2022-09-22 61/week @ 2022-09-29 42/week @ 2022-10-06 82/week @ 2022-10-13 175/week @ 2022-10-20 102/week @ 2022-10-27 97/week @ 2022-11-03 113/week @ 2022-11-10 61/week @ 2022-11-17 93/week @ 2022-11-24

368 downloads per month

Apache-2.0

4MB
601 lines

PDQ Hash

This hash is a pure Rust implementation of the PDQ algorithm described here.

Calculation

The PDQ algorithm was developed and open-sourced by Facebook (now Meta) in 2019. It specifies a transformation which converts images into a binary format ('PDQ Hash') whereby 'perceptually similar’ images produce similar outputs. It was designed to offer an industry standard for representing images to collaborate on threat mitigation.

Obtain PDQ Hash of an image

Comparing two images reduces to computing distance (for example, Hamming distance) between their representations, or as % bit similarity. Comparing two bit strings

(16 bits are just used here for easier interpretation; PDQ hashes represent 256 bits)

Consider additional image transformations

Additionally, PDQ hashes for rotations and mirrors of the original image can be inferred efficiently, by just manipulating the Discrete Cosine Transform created in latter stages of processing. Comparing two bit strings Example: PDQ Hash of mirrored original image only requires manipulation of the transform

DCT Manipulation needed for corresponding PDQ Hash

  • Mirrored Y -> Negate alternate columns
  • Mirrored X -> Negate alternate rows
  • Mirrored Main Diagonal -> Transpose
  • Mirrored Off Diagonal -> Negate off-diagonal, transpose
  • Rotated 90 -> Negate alternate columns, transpose
  • Rotated 180 -> Negate off-diagonal
  • Rotated 270 -> Negate alternate rows, transpose

Offering similarity resilience

The resulting hashes are resilient to certain transformations, some more so than others, to detect additional attempted manipulation. Generally, images retaining overall structure are more resilient than changes to pixel positions and larger areas of pixel change. Obtain PDQ Hash of an image

Transformations that result in similar hashes:

  • File format change
  • Quality reduction
  • Light crops and shifts
  • Rotations (when additional hashes compared)
  • Resizing
  • Light watermarks
  • Mirroring (when additional hashes compared)
  • Noise or filter applied
  • Light logos

Dependencies

~5MB
~59K SLoC