#duplicate #similarity #similar #compare

bin+lib near-facsimile

Find similar or identical text files in a directory

10 releases (stable)

1.0.7 Jul 27, 2023
1.0.5 Jan 26, 2023
1.0.2 Oct 25, 2022
1.0.1 Aug 24, 2022
0.8.2 Aug 17, 2022

#272 in Text processing

Download history 4/week @ 2023-11-02 11/week @ 2023-11-09 1/week @ 2023-11-16 12/week @ 2023-11-23 42/week @ 2023-11-30 10/week @ 2023-12-14 20/week @ 2023-12-21 10/week @ 2023-12-28 10/week @ 2024-01-25 12/week @ 2024-02-01 11/week @ 2024-02-08 303/week @ 2024-02-15

336 downloads per month


496 lines


Find similar or identical text files in a directory.

Rust tests dependency status

  • reasonable facsimile, noun

    : a copy that is not exact but is fairly close

    The house is a reasonable facsimile of his original home.

    —sometimes used in a joking way

    I can speak French, or at least a reasonable facsimile of French.

    “reasonable facsimile,” Merriam-Webster.com Dictionary


  • On Fedora, RHEL, or CentOS, install this program from the Copr repository:

    1. Enable the repository:

      # dnf copr enable mareksu/near-facsimile
    2. Install the package:

      # dnf install near-facsimile
  • On macOS, use the Homebrew package manager:

    1. Install the Homebrew package manager as described at https://brew.sh/.

    2. Install the package:

      $ brew install msuchane/repo/near-facsimile
  • To install from source on any system, use the Cargo package manager:

    1. Install Cargo as described at https://rustup.rs/.

    2. Install the package:

      $ cargo install near-facsimile
  • To install near-facsimile as a container, use Docker or Podman.

    1. Download the image:

      $ docker pull quay.io/msuchane/near-facsimile:latest
    2. Run the image:

      $ docker run -it -v .:/mnt:Z msuchane/near-facsimile near-facsimile
    3. Optionally, save the command as an alias in your shell configuration.


  • Recursively examine a directory of text files:

    dir-with-text]$ near-facsimile

    The program continually prints out file information to the terminal.

    Finally, it can save all statistics sorted by file similarity as a CSV or JSON file.


The following options are available:

Specifying the documentation directory

$ near-facsimile --path <path-to-directory>

Saving the the results as a CSV table or a JSON file

Optionally, you can save the file similarities above the set threshold as CSV or JSON:

$ near-facsimile --csv <path-to-file>
$ near-facsimile --json <path-to-file>

Setting the lowest reported similarity threshold

The tool only reports files that are similar over a certain threshold. By default, the threshold is 85.0, or 85% similar.

$ near-facsimile --threshold=<85.0>

Disregarding certain lines in files

You can configure the file comparison such that it skips all lines that match your regular expressions. The comparison is the calculated from the remaining lines, which match none of the regular expressions.

For example, skip all lines that start with //:

$ near-facsimile --skip-lines '^//'

Switching to a faster, less accurate comparison

By default, the tool uses the Levenshtein metric, which is accurate but rather slow. You can instead compare files using the Jaro metric, which finishes in around half the time, but produces less accurate statistics.

$ near-facsimile --fast

If you specify the --fast option twice (-ff), the tool uses the relatively rudimentary but very fast trigram comparison instead:

$ near-facsimile --fast --fast


~341K SLoC