Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

github.com

Post reply on HN
Print view View original

11–20 of 66 posts

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#11

Post by d76d6776yudsy »

watermelon59 wrote: post #6

Years ago I worked with image processing and thought of writing a CLI image diff tool. I then thought meh, anyone can quickly roll their own. Turns out it seems to be something people care about. I should just start writing the CLI tools I think of, without dismissing their utility.

mind blown

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#13

Post by formerly_proven »

ajfjrbfbf wrote: post #2

Why not use the GPU? This is exactly the kind of tasks GPUs were designed for. Gimp can do this in real time using the difference layer mode.

The bottleneck is almost certainly decoding the image (PNGs are zlib compressed, which will be going at something like ~300 MB/s plus the filter stuff PNG does), not comparing raw pixels (which you should be able to do at memory bandwidth, so something in excess of 10 GP/s).

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#14

Post by jihchi »

dheera wrote: post #4

I'm finding "gm compare" to be faster. $ time ./imgdiff cypress.png cypress.png Success! Images are equal. 3.22user 0.10system 0:00.50elapsed 661%CPU (0avgtext+0avgdata 478524maxresident)k 0inputs+0outputs (0major+48339minor)pagefaults 0swaps $ time gm compare -highlight-style assign -highlight-color purple -file diff.png cypress.png cypress.png 1.99user 0.21system 0:02.21elapsed 99%CPU (0avgtext+0avgdata 874948maxre…

Were you comparing two identical image? Could you try two different image?

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#15

Post by viraptor »

ajfjrbfbf wrote: post #2

Why not use the GPU? This is exactly the kind of tasks GPUs were designed for. Gimp can do this in real time using the difference layer mode.

Even before you get into GPU, it looks like you could do the normalisation here https://github.com/n7olkachev/imgdiff/blob/master/pkg/yiq/de... using _mm_mul_ps or _mm256_mul_ps depending on availability instead of standard math. (CPU based vector processing)

There's also bound to be some vector compare trick as well which falls back to pixel-by-pixel only for differences.

I think there are quite a few simple tricks you could apply here to get a bit more performance... Starting with sacrificing some memory and decoding a chunk of image at a time rather than calling .At(x,y) every time. And maybe improving local cache by dividing the image in `cpu` parts rather than having the work interleaved.

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#16

Post by noahtallen »

jihchi wrote: post #14
dheera wrote: post #4

I'm finding "gm compare" to be faster. $ time ./imgdiff cypress.png cypress.png Success! Images are equal. 3.22user 0.10system 0:00.50elapsed 661%CPU (0avgtext+0avgdata 478524maxresident)k 0inputs+0outputs (0major+48339minor)pagefaults 0swaps $ time gm compare -highlight-style assign -highlight-color purple -file diff.png cypress.png cypress.png 1.99user 0.21system 0:02.21elapsed 99%CPU (0avgtext+0avgdata 874948maxre…

Were you comparing two identical image? Could you try two different image?

To really see how these algorithms perform, you should compare the worst case for the algorithm. In order to see if a picture is identical, you’d have to loop through every single pixel, which is the worst case. But you can shortcut on the first pixel which is different, which I imagine happens pretty quickly on many generic pictures. So testing performance with two different images is probably not best for comparison.

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#17

Post by kaetemi »

ajfjrbfbf wrote: post #2

Why not use the GPU? This is exactly the kind of tasks GPUs were designed for. Gimp can do this in real time using the difference layer mode.

Considering the source material is a file, that needs to be loaded up by the CPU first, and the operation itself is dead simple, the real constraint is RAM bandwidth.

I would guess that the time of getting the data out to the GPU and back, is about the same as just calculating the difference on the CPU.

So, the quickest solution is probably to use the CPUs vector operations, and read ahead into the registers so you can exhaust the memory bandwidth fully.

Re: Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

#19

Post by amelius »

kaetemi wrote: post #17
ajfjrbfbf wrote: post #2

Why not use the GPU? This is exactly the kind of tasks GPUs were designed for. Gimp can do this in real time using the difference layer mode.

Considering the source material is a file, that needs to be loaded up by the CPU first, and the operation itself is dead simple, the real constraint is RAM bandwidth. I would guess that the time of getting the data out to the GPU and back, is about the same as just calculating the difference on the CPU. So, the quickest solution is probably to use the CPUs vector operations, and read ahead into the registers so you c…

I recently tried this with a 19 megabyte .tiff file in PyTorch. Uploading the tensor took (i think) around 1 or 2 seconds, wall-clock time. So I quickly dismissed the GPU solution to my problem. But I'm still wondering... is this normal?
Post reply on HN