Show HN: Neural Image Compression Demo • Hacker News Board

Re: Show HN: Neural Image Compression Demo

#31

Post by tonic_section » Sat, Sep 12, 2020, 4:55 PM UTC

fireattack wrote: post #29

The result seems pretty poor to me? (I just use the exmaple image that is already in the notebook) Original: https://i.imgur.com/Q66mHTD.png Result: https://i.imgur.com/4R6qn8e.png There are lots of random spots on the image, and the brightness level changes totally. Sure, 5232 kB to 124 kB is impressive, but people would probably prefer a badly compressed JPEG over this, since at least JPEG artifact is predictable (…

Hey, thanks for bringing the brightness issue to my attention - turns out I wasn't normalizing the output correctly - I just pushed a fix and the output images don't have the brightness change now.

As for the random spots, that's an artifact of the entropy coding algorithm. In principle this is lossless but there is some distortion because I'm using a custom vectorized version of an rANS encoder and it's hard to encode overflow values in a vectorized fashion, I'm working on this though. If you can live with really slow decoding times (2-3mins) then you can disable vectorization to eliminate these small imperfections entirely.

As for the comparison to the official model, that's mainly because of compute constraints v. Google (this is just my weekend project). My model uses a smaller architecture and was trained for only 4e5 steps versus the 2e6 steps they reported in the paper - even then it took 4+ days on AWS! The model is also trained on the Openimages dataset, which is presumably much smaller and more noisy than the massive internal dataset Google used.

Re: Show HN: Neural Image Compression Demo

#32

Post by garblegarble » Sat, Sep 12, 2020, 6:01 PM UTC

godelski wrote: post #12

garblegarble wrote: post #5

Earlier quoted context omitted.

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

The model uses a GAN which does not learn the exact PDF. So not lossless, but as you can see from the images it gets extremely visually accurate results. From the README > The generator is trained to achieve realistic and not exact reconstruction. It may synthesize certain portions of a given image to remove artifacts associated with lossy compression. Therefore, in theory images which are compressed and decoded may…

>The model uses a GAN which does not learn the exact PDF. So not lossless, but as you can see from the images it gets extremely visually accurate results.

Yes, I understand this is a lossy compression method - what I was proposing is to have the compressor as a final pass take the predicted output image, and subtract it from the original pixels. This gives you a delta between the predicted image and the original image. You can then compress that delta losslessly, and store it alongside the output of this model - if the predicted image is close enough to the original image then you've significantly reduced the amount of entropy in the delta, making it highly compressible.

This is how some domain-specific lossless compression algorithms work, e.g. DTS-HD Master Audio

Re: Show HN: Neural Image Compression Demo

#33

Post by tgsovlerkhgsel » Sat, Sep 12, 2020, 6:44 PM UTC

tonic_section wrote: post #26

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

Sorry, looks like both GDrive and Zenodo have exceeded the temporary download quotas, so the model checkpoints aren't available currently... If anyone has any solutions on how to publicly host model weights (~2 GB) please let me know!

I would recommend to link to a site where some example images can be easily compared (ideally with a viewer that offers toggling between them in-place to make it easy to see the differences), instead of directly linking to a colab that does heavy computations.

I assume most people just want to see the images, forcing them to recompute them is a waste of resources. Even just storing a version of the colab with the results present would help a lot.

Re: Show HN: Neural Image Compression Demo

#34

Post by tgsovlerkhgsel » Sat, Sep 12, 2020, 6:48 PM UTC

tonic_section wrote: post #28

codewithcheese wrote: post #27

Earlier quoted context omitted.

torrent? If you create a torrent I can seed it for a while

I eventually shifted the models to S3, but thanks for the offer.

How do you handle the traffic? If every reader who clicks the link and runs the colab costs you 15 cents for traffic, that's got to get expensive unless you have some sort of "free traffic" deal or someone else is paying for it?

Re: Show HN: Neural Image Compression Demo

#35

Post by wumms » Sat, Sep 12, 2020, 7:24 PM UTC

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

> or notice something weird

> [4]: Sample reconstructions

The text in the reconstructed image in the third row looks different, the word phonomat is quite garbled, information looks a bit funny.

Re: Show HN: Neural Image Compression Demo

#36

Post by tonic_section » Sat, Sep 12, 2020, 7:27 PM UTC

tgsovlerkhgsel wrote: post #34

tonic_section wrote: post #28

Earlier quoted context omitted.

I eventually shifted the models to S3, but thanks for the offer.

How do you handle the traffic? If every reader who clicks the link and runs the colab costs you 15 cents for traffic, that's got to get expensive unless you have some sort of "free traffic" deal or someone else is paying for it?

I think S3 permits up to 20k requests before they start billing IIRC.

Re: Show HN: Neural Image Compression Demo

#37

Post by tonic_section » Sat, Sep 12, 2020, 7:31 PM UTC

wumms wrote: post #35

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

> or notice something weird > [4]: Sample reconstructions The text in the reconstructed image in the third row looks different, the word phonomat is quite garbled, information looks a bit funny.

Yeah, high frequency detail such as facial features for faraway figures or text tend to get washed out after compression - this is probably due to a couple reasons: 1) The training dataset contains relatively few pictures including text, 2) high-frequency detail is too expensive to encode and the model learns to forgo encoding this in favor of more 'important' features such as shapes, colors, etc.

Re: Show HN: Neural Image Compression Demo

#38

Post by fireattack » Sat, Sep 12, 2020, 10:53 PM UTC

tonic_section wrote: post #31

fireattack wrote: post #29

The result seems pretty poor to me? (I just use the exmaple image that is already in the notebook) Original: https://i.imgur.com/Q66mHTD.png Result: https://i.imgur.com/4R6qn8e.png There are lots of random spots on the image, and the brightness level changes totally. Sure, 5232 kB to 124 kB is impressive, but people would probably prefer a badly compressed JPEG over this, since at least JPEG artifact is predictable (…

Hey, thanks for bringing the brightness issue to my attention - turns out I wasn't normalizing the output correctly - I just pushed a fix and the output images don't have the brightness change now. As for the random spots, that's an artifact of the entropy coding algorithm. In principle this is lossless but there is some distortion because I'm using a custom vectorized version of an rANS encoder and it's hard to enco…

Thank you!

Re: Show HN: Neural Image Compression Demo

#39

Post by fireattack » Sun, Sep 13, 2020, 1:05 AM UTC

tonic_section wrote: post #31

fireattack wrote: post #29

The result seems pretty poor to me? (I just use the exmaple image that is already in the notebook) Original: https://i.imgur.com/Q66mHTD.png Result: https://i.imgur.com/4R6qn8e.png There are lots of random spots on the image, and the brightness level changes totally. Sure, 5232 kB to 124 kB is impressive, but people would probably prefer a badly compressed JPEG over this, since at least JPEG artifact is predictable (…

Hey, thanks for bringing the brightness issue to my attention - turns out I wasn't normalizing the output correctly - I just pushed a fix and the output images don't have the brightness change now. As for the random spots, that's an artifact of the entropy coding algorithm. In principle this is lossless but there is some distortion because I'm using a custom vectorized version of an rANS encoder and it's hard to enco…

Just curious, is the change on the model side? Since I didn't see much relevant in the notebook's rev history [1].

[1] https://colab.research.google.com/github/Justin-Tan/high-fid...

Re: Show HN: Neural Image Compression Demo

#40

Post by tgsovlerkhgsel » Sun, Sep 13, 2020, 10:35 PM UTC

tonic_section wrote: post #36

tgsovlerkhgsel wrote: post #34

Earlier quoted context omitted.

How do you handle the traffic? If every reader who clicks the link and runs the colab costs you 15 cents for traffic, that's got to get expensive unless you have some sort of "free traffic" deal or someone else is paying for it?

I think S3 permits up to 20k requests before they start billing IIRC.

I hope I'm wrong, but I believe that's for the cost of the request processing, not covering the traffic.

The free tier for traffic is "15GB of Data Transfer Out", after that it's 9 cents per GB. https://aws.amazon.com/s3/pricing/?nc1=h_ls (under "Data transfer"). Check your AWS bill!