Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differentiable Dithering

peterstefek.me

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Differentiable Dithering

#31

Post by MiroF »

underanalyzer wrote: post #29
londons_explore wrote: post #2

> A pipedream would be an entirely differentiable image compression pipeline where all the steps can be fine tuned together to optimize a particular image with respect to any differentiable loss function. Neural Image Compression? https://arxiv.org/abs/1908.08988

Yea neural image compression looks pretty neat! The reason I bring up jpeg is because it's so well established and if you come up with a more optimized jpeg (which I'm not really convinced is possible, again a pipedream) you don't have to force people to transition to a new image format. In the end methods like neural style or whatever comes after are probably the better pick but there is a transition period.

> more optimized jpeg

The thing about compression is that there is no single "more optimized" knob - there's a bunch of different tradeoffs.

Want a compression algo that can compress existing images to smaller sizes than JPEG? You can already do that with neural image compression. Want a compression algo that can decode that compressed image in 0.01 seconds? You need JPEG.

Re: Differentiable Dithering

#32

Post by MiroF »

amelius wrote: post #11
londons_explore wrote: post #2

> A pipedream would be an entirely differentiable image compression pipeline where all the steps can be fine tuned together to optimize a particular image with respect to any differentiable loss function. Neural Image Compression? https://arxiv.org/abs/1908.08988

I fear that you might end up with hallucinations in your images ...

It's not trained to generate faces, so I don't think so.

Re: Differentiable Dithering

#33

Post by MiroF »

londons_explore wrote: post #2

> A pipedream would be an entirely differentiable image compression pipeline where all the steps can be fine tuned together to optimize a particular image with respect to any differentiable loss function. Neural Image Compression? https://arxiv.org/abs/1908.08988

> entirely differentiable image compression pipeline

Depending on what is meant by entirely differentiable, this might be impossible without relaxation. ie. you can't differentiate through the quantization step

Re: Differentiable Dithering

#34

Post by aharris6 »

anilgulecha wrote: post #3

I've recently done a few things around dithering, and found this site good to experiement: https://ditherit.com/ It's open source: https://github.com/alexharris/ditherit-v2

Hi, thanks for using Dither it! I built it, and would gladly receive any feedback, suggestions or further praise!

Re: Differentiable Dithering

#35

Post by underanalyzer »

MiroF wrote: post #31
underanalyzer wrote: post #29

Earlier quoted context omitted.

Yea neural image compression looks pretty neat! The reason I bring up jpeg is because it's so well established and if you come up with a more optimized jpeg (which I'm not really convinced is possible, again a pipedream) you don't have to force people to transition to a new image format. In the end methods like neural style or whatever comes after are probably the better pick but there is a transition period.

> more optimized jpeg The thing about compression is that there is no single "more optimized" knob - there's a bunch of different tradeoffs. Want a compression algo that can compress existing images to smaller sizes than JPEG? You can already do that with neural image compression. Want a compression algo that can decode that compressed image in 0.01 seconds? You need JPEG.

Sorry I could have been more specific. By more optimized jpeg I meant better perceptual quality (again subjective) within the confines of what a jpeg decoder could understand.

To co opt your knobs analogy I imagine each of the steps of a complex image compression pipeline comes with its own knobs each with its own tradeoffs. The dream here would be to tune all those knobs at the same time to optimize some sense of quality in a particular image. Of course huge disclaimer I’m not an image or signal processing expert. It’s also very possible that these “knobs” have been tuned well enough so that even if we optimized them for a specific image the quality difference would not be noticeable.

Re: Differentiable Dithering

#36

Post by dgant »

A great read on dithering: Lucas Pope's development blogs while working on Return of the Obra Dinn.

It's an incredible dive into how he created the game's remarkable and unique look, featuring a wonderful and unexpected mathematical contribution from a forum member. If you're not familiar with the game peek at a trailer to see what an achievement it was.

https://forums.tigsource.com/index.php?topic=40832.msg136374...

Re: Differentiable Dithering

#37

Post by underanalyzer »

aharris6 wrote: post #34
anilgulecha wrote: post #3

I've recently done a few things around dithering, and found this site good to experiement: https://ditherit.com/ It's open source: https://github.com/alexharris/ditherit-v2

Hi, thanks for using Dither it! I built it, and would gladly receive any feedback, suggestions or further praise!

Wow exciting! I tried to use ditherit to get some baseline comparison images for my post! Is there anyway to both control the number of colors in the palette and have it auto pick colors at the same time?

Re: Differentiable Dithering

#38

Post by tonic_section »

MiroF wrote: post #30
tonic_section wrote: post #17

Earlier quoted context omitted.

Unfortunately you wouldn't have any guarantees on the output of any particular image though, just some reassurances about the expected behaviour over the training set.

> any guarantees on the output of any particular image though You don't have any guarantees with this non-convex optimization. I think most of these methods would work OK on out-of-domain data.

In terms of the decoded image, yes - it's very unlikely you would get something substantially different from the original image. But in terms of the bitrate it's not hard to find examples where the compressed bitrate can be several standard deviations above the average bitrate on the training set - see e.g. the last example here: https://github.com/Justin-Tan/high-fidelity-generative-compr...

(Lossy) neural compression methods may also synthesize small portions of an image to avoid compression artefacts associated with standard image codecs, so should definitely not be used in sensitive applications where small details can make a big difference such as security imaging, guarantees or none.

Re: Differentiable Dithering

#39

Post by tonic_section »

MiroF wrote: post #33
londons_explore wrote: post #2

> A pipedream would be an entirely differentiable image compression pipeline where all the steps can be fine tuned together to optimize a particular image with respect to any differentiable loss function. Neural Image Compression? https://arxiv.org/abs/1908.08988

> entirely differentiable image compression pipeline Depending on what is meant by entirely differentiable, this might be impossible without relaxation. ie. you can't differentiate through the quantization step

There are a couple of solutions which work empirically - as you mentioned, one solution is a dithering-like differentiable relaxation where uniform noise is added, which simulates quantization, or just to ignore the quantization operation when taking gradients, essentially treating it as an identity operation in the backward pass.

Re: Differentiable Dithering

#40

Post by aharris6 »

underanalyzer wrote: post #37
aharris6 wrote: post #34

Earlier quoted context omitted.

Hi, thanks for using Dither it! I built it, and would gladly receive any feedback, suggestions or further praise!

Wow exciting! I tried to use ditherit to get some baseline comparison images for my post! Is there anyway to both control the number of colors in the palette and have it auto pick colors at the same time?

That feature does not currently exist, but that is a great idea. I have added it to the list (which is just the github issue tracker, and contains no other items). I like that it auto-analyzes the palette when the images loads without requiring any further user input, but maybe an option after it loads to "auto-detect X colors" as a little dropdown thingie.
Post reply on HN