Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compressing Images with Neural Networks

mlumiste.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Compressing Images with Neural Networks

#61

Post by ufocia »

esafak wrote: post #43
userbinator wrote: post #37

Earlier quoted context omitted.

That makes the codec not safe for archiving, only for distribution. Could you explain what you mean by "not safe for archiving"? The standard is published and there are multiple implementations, some of which are open-source. There is no danger of it being a proprietary format with no publicly available specification.

I was not thinking of J2K as being problematic for archiving but these new neural codecs. My point being that performance is only one of the criteria used to evaluate a codec.

Royalty costs are often the other.

Re: Compressing Images with Neural Networks

#62

Post by 3abiton »

dheera wrote: post #45
Dwedit wrote: post #24

There was an earlier article (Sep 20, 2022) about using the Stable Diffusion VAE to perform image compression. Uses the VAE to change from pixel space to latent space, dithers the latent space down to 256 colors, then when it's time to decompress it, it de-noises that. https://pub.towardsai.net/stable-diffusion-based-image-compr... HN discussion: https://news.ycombinator.com/item?id=32907494

I've done a bunch of experiments on my own on the Stable Diffusion VAE. Even when going down to 4-6 bits per latent space pixel the results are surprisingly good. It's also interesting what happens if you ablate individual channels; ablating channel 0 results in faithful color but shitty edges, ablating channel 2 results in shitty color but good edges, etc. The one thing it fails catastrophically on though is small t…

How does the type of image (Anime, vs Photo realistic, vs Painting vs etc .m) affect the compression results? Is there a noticable difference?

Re: Compressing Images with Neural Networks

#63

Post by ufocia »

amelius wrote: post #9

How do we know we don't get hands with 16 fingers?

How do we know whether it's an image with 16 fingers or it just looks like 16 fingers to us?

I looked at the bear example above and I could see how either the AI thought that there was an animal face embedded in the fur or we just see the face in the fur. We see all kinds of faces on toast even though neither the bread slicers nor the toasters intend to create them.

Re: Compressing Images with Neural Networks

#64

Post by fl7305 »

_kb wrote: post #51
StiffFreeze9 wrote: post #29

How badly will its lossy-ness change critical things? In 2013, there were Xerox copiers with aggressive compression that changed numbers, https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...

The suitable lossy-ness (of any compression method) is entirely dependant on context. There is no one size fits all approach for all uses cases. One key item with emerging 'AI compression' techniques is the information loss is not deterministic which somewhat complicates assessing suitability.

> the information loss is not deterministic

It is technically possible to make it deterministic.

The main reason you don't deterministic outputs today is that Cuda/GPU optimizations make the calculations run much faster if you let them be undeterministic.

The internal GPU scheduler will then process things in the order it thinks is fastest.

Since floating point is not associative, you can get different results for (a + (b + c)) and ((a + b) + c).

Re: Compressing Images with Neural Networks

#65

Post by fieldcny »

lifthrasiir wrote: post #42
qrian wrote: post #35

Earlier quoted context omitted.

The mentioned Xerox copier incident was not an OCR failure, but the copier actively changed the numbers in the original image due to its image compression algorithm.

Any lossy compressor changes the original image for better compression at expense of the perfect accuracy.

The word perfect is misplaced, the trade off is size vs fidelity (aka accuracy)

Re: Compressing Images with Neural Networks

#67

Post by jasonjmcghee »

davidbarker wrote: post #27
jfdi wrote: post #10

Anyone know of open models useful (and good quality) for going the other way? I.e., Input is a 800x600 jpg and output is 4k version.

Magnific.ai ( https://magnific.ai ) is a paid tool that works well, but it is expensive. However, this weekend someone released an open-source version which has a similar output. ( https://replicate.com/philipp1337x/clarity-upscaler ) I'd recommend trying it. It takes a few tries to get the correct input parameters, and I've noticed anything approaching 4× scale tends to add unwanted hallucinations. For example, I ha…

Both of these links to replicate 404 for me

Re: Compressing Images with Neural Networks

#68

Post by davidbarker »

jasonjmcghee wrote: post #67
davidbarker wrote: post #27

Earlier quoted context omitted.

Magnific.ai ( https://magnific.ai ) is a paid tool that works well, but it is expensive. However, this weekend someone released an open-source version which has a similar output. ( https://replicate.com/philipp1337x/clarity-upscaler ) I'd recommend trying it. It takes a few tries to get the correct input parameters, and I've noticed anything approaching 4× scale tends to add unwanted hallucinations. For example, I ha…

Both of these links to replicate 404 for me

Ah, the user changed their username.

https://replicate.com/philz1337x/clarity-upscaler https://replicate.com/philz1337x/multidiffusion-upscaler

Re: Compressing Images with Neural Networks

#69

Post by thomastjeffery »

StiffFreeze9 wrote: post #29

How badly will its lossy-ness change critical things? In 2013, there were Xerox copiers with aggressive compression that changed numbers, https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...

Lossy compression has the same problem it has always had: lossy metadata.

The contextual information surrounding intentional data loss needs to be preserved. Without that context, we become ignorant of the missing data. Worst case, you get replaced numbers. Average case, you get lossy->lossy transcodes, which is why we end up with degraded content.

There are only two places to put that contextual information: metadata and watermarks. Metadata can be written to a file, but there is no guarantee it will be copied with that data. Watermarks fundamentally degrade the content once, and may not be preserved in derivative works.

I wish that the generative model explosion would result in a better culture of metadata preservation. Unfortunately, it looks like the focus is on metadata instead.

Re: Compressing Images with Neural Networks

#70

Post by dheera »

3abiton wrote: post #62
dheera wrote: post #45

Earlier quoted context omitted.

I've done a bunch of experiments on my own on the Stable Diffusion VAE. Even when going down to 4-6 bits per latent space pixel the results are surprisingly good. It's also interesting what happens if you ablate individual channels; ablating channel 0 results in faithful color but shitty edges, ablating channel 2 results in shitty color but good edges, etc. The one thing it fails catastrophically on though is small t…

How does the type of image (Anime, vs Photo realistic, vs Painting vs etc .m) affect the compression results? Is there a noticable difference?

I haven't noticed much difference between these. They're all well-represented in the VAE training set.
Post reply on HN