Diff-pdf: tool to visually compare two PDFs • Hacker News Board

Re: Diff-pdf: tool to visually compare two PDFs

#41

Post by strangus » Tue, Jul 02, 2024, 2:27 PM UTC

https://10052.ai has a tool that will visually compare documents(pdfs, doc, image,etc) and cluster them together. It works amazingly well.

Re: Diff-pdf: tool to visually compare two PDFs

#42

Post by ydant » Tue, Jul 02, 2024, 2:30 PM UTC

pmarreck wrote: post #20

thibaut_barrere wrote: post #9

I have been using this in a CI pipeline to maintain a business-critical PDF generation (healthcare) app (started circa 2010 I think), here is the RSpec helpers I'm using: https://gist.github.com/thbar/d1ce2afef68bf6089aeae8d9ddc05d... The code contains git-stored reference PDFs, and the test suite re-generate them and assert that nothing has changed. Helped a lot to audit visual changes, or PDF library upgrades!

could you not just compare the source (or perhaps even the hash) of the PDF and assert on that?

I use some custom tools for PDF comparison (visual, textual, and perceptual hash) for my personal records/accounting purposes.

A number of the financial and medical institutions I deal with re-generate PDFs every time you request them, but the content is 99-100% identical. Sometimes just a date changes. So I use a perceptual hash and content comparison to automate detecting truly new documents vs. ones that are only slightly changed.

Re: Diff-pdf: tool to visually compare two PDFs

#43

Post by ydant » Tue, Jul 02, 2024, 2:31 PM UTC

Related - this might be helpful to someone.

ImageMagick can do a visual PDF compare:

    magick compare -density "$DENSITY" -background white "$1[0]" "$2[0]" "$TMP"

(density = 100, $1 and $2 are the filenames to compare, $TMP the output file)

You need to do some work to support multiple pages, so I use this script:

https://gist.github.com/mbafford/7e6f3bef20fc220f68e467589bb...

This also uses `imgcat` to show the difference directly in the terminal.

You can also use ImageMagick get a perceptual hash difference using something like:

    convert -metric phash "$1" null: "$2" -compose Difference -layers composite -format '%[fx:mean]\n' info:

I use the fact you can configure git to use custom diff tools and take advantage of this with the following in my .gitconfig:

    [diff "pdf"]
        command = ~/bin/git-diff-pdf

And in my .gitattributes I enable the above with:

    *.pdf binary diff=pdf

~/bin/git-diff-pdf does a diff of the output of `pdftotext -layout` (from poppler) and also runs pdf-compare-phash.

To use this custom diff with `git show`, you need to add an extra argument (`git show --ext-diff`), but it uses it automatically if running `git diff`.

Re: Diff-pdf: tool to visually compare two PDFs

#44

Post by akasakahakada » Tue, Jul 02, 2024, 2:31 PM UTC

ant6n wrote: post #18

akasakahakada wrote: post #14

Use this to compare university textbook edition 8 and 9 before buying.

Uh how can you compare without buying? Or put another way, why buy if you can compare?

libgen exist bro

Re: Diff-pdf: tool to visually compare two PDFs

#45

Post by pivo » Tue, Jul 02, 2024, 2:35 PM UTC

tomwheeler wrote: post #38

In a previous job, I had to validate the output of an unreliable production publishing system, so I tested dozens of PDF comparison tools available at the time. The best I found was called Delta Walker. It was proprietary commercial Mac-only software, but reasonably inexpensive, accurate, and could handle long PDFs with lots of graphics well. I remember evaluating this diff-pdf tool and finding that it fell short in…

It looks like Delta Walker's added Windows and Linux support: https://www.deltawalker.com/download

Re: Diff-pdf: tool to visually compare two PDFs

#46

Post by fwn » Tue, Jul 02, 2024, 2:37 PM UTC

I really like the overlay view and that it is not cloud based. Will try to test it at work.

I rely heavily on PDF comparison via PDF-XChange Editor, which is accurate for text, but often has trouble highlighting visual changes correctly.

Re: Diff-pdf: tool to visually compare two PDFs

#47

Post by hipnoizz » Tue, Jul 02, 2024, 2:38 PM UTC

Rinzler89 wrote: post #37

mikeyinternews wrote: post #35

You can do this with Beyond Compare (it's not free, but not very expensive either) https://www.scootersoftware.com/

Beyond Compare is one of those priceless tools I pay for myself instead of waiting for my employer to pay for it. Price/functionality wise it's worth its weight in gold, it's cross platform, and its licensing is very liberal. There's just no FOSS compare tools out there that can match BC.

What are BC features that you find to be so great?

I'm genuinely curious - I heard of lot of BC being 'the tool' for diffing. I'm used to Meld, but my current employee has a pretty strict policy which tools could be used so at some point I've managed a licence for some older version of BC. But for some reason I've found its UI/the way it works a bit less optimal that I was accustomed for. Since I'm using that primarily for text diffs these day I usually use a diff tool from IntelliJ Idea (I have Idea open all the time).

Re: Diff-pdf: tool to visually compare two PDFs

#48

Post by N0b8ez » Tue, Jul 02, 2024, 2:39 PM UTC

akasakahakada wrote: post #44

ant6n wrote: post #18

Earlier quoted context omitted.

Uh how can you compare without buying? Or put another way, why buy if you can compare?

libgen exist bro

But then why would you need to buy it?

Re: Diff-pdf: tool to visually compare two PDFs

#49

Post by bigfatfrock » Tue, Jul 02, 2024, 3:12 PM UTC

ydant wrote: post #43

Related - this might be helpful to someone. ImageMagick can do a visual PDF compare: magick compare -density "$DENSITY" -background white "$1[0]" "$2[0]" "$TMP" (density = 100, $1 and $2 are the filenames to compare, $TMP the output file) You need to do some work to support multiple pages, so I use this script: https://gist.github.com/mbafford/7e6f3bef20fc220f68e467589bb... This also uses `imgcat` to show the differe…

Next level, especially with the git attribute calls, well played.

I'm still blown away how powerful imagemagick is after using it for a decade or two, what an inspiring piece of open source software.

Re: Diff-pdf: tool to visually compare two PDFs

#50

Post by pixelmonkey » Tue, Jul 02, 2024, 3:18 PM UTC

ck_one wrote: post #34

Can anyone recommend a method to deduplicate pdfs? The hash is often different but the content and meta data is 99.99% the same.

You might want strip metadata before doing a comparison, using exiftool. Even though exiftool was originally written for EXIF metadata on JPGs, these days, it supports a lot of metadata standards, including PDF. This command will do it assuming you set filename=`basename your.pdf .pdf`:

    exiftool -all= -o ${filename}.stripped.pdf ${filename}.pdf

That won't help you with small differences in the contents, but might help with small differences in metadata. Running `md5sum` on the stripped PDF should give more reliable dedupe results.

I was recently working on a similar problem for JPG, RAW, and MP4 files (photo/video backup) so it is fresh in my mind.