Show HN: Neural Image Compression Demo • Hacker News Board

Re: Show HN: Neural Image Compression Demo

#11

Post by londons_explore » Sat, Sep 12, 2020, 9:23 AM UTC

garblegarble wrote: post #5

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

I suspect if lossless reconstruction was your goal, you would want a different architecture. You would want the model to give you a conditional probability distribution for each pixel, conditioned on all previous pixels, so you could use a regular entropy coder to encode exact data.

Re: Show HN: Neural Image Compression Demo

#12

Post by godelski » Sat, Sep 12, 2020, 9:24 AM UTC

garblegarble wrote: post #5

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

The model uses a GAN which does not learn the exact PDF. So not lossless, but as you can see from the images it gets extremely visually accurate results.

From the README

> The generator is trained to achieve realistic and not exact reconstruction. It may synthesize certain portions of a given image to remove artifacts associated with lossy compression. Therefore, in theory images which are compressed and decoded may be arbitrarily different from the input. This precludes usage for sensitive applications. An important caveat from the authors is reproduced here:

> "Therefore, we emphasize that our method is not suitable for sensitive image contents, such as, e.g., storing medical images, or important documents."

Re: Show HN: Neural Image Compression Demo

#13

Post by s-macke » Sat, Sep 12, 2020, 9:27 AM UTC

tonic_section wrote: post #10

browserface wrote: post #7

Damn I got an error at cell 19 otherwise seemed to work

What was the error? I tried to make the demo notebook as robust as possible - you should be able to execute all cells in sequence once then execute cells out of sequence etc. without trouble, but it's hard to legislate for errors in Jupyter-like notebooks sometimes.

In the step

# Setup model

I get an error in the function call 'prepare_model'

UnpicklingError: invalid load key, '<'.

Re: Show HN: Neural Image Compression Demo

#14

Post by underaxon » Sat, Sep 12, 2020, 9:30 AM UTC

tonic_section wrote: post #10

browserface wrote: post #7

Damn I got an error at cell 19 otherwise seemed to work

What was the error? I tried to make the demo notebook as robust as possible - you should be able to execute all cells in sequence once then execute cells out of sequence etc. without trouble, but it's hard to legislate for errors in Jupyter-like notebooks sometimes.

The models aren't downloading correctly. The content of the '*.pt' files says 'Google Drive - Quota exceeded'. I guess too many people have tried downloading the files from your drive.

One solution is to download (and upload to Colab) the models manually in /content/checkpoint/

Re: Show HN: Neural Image Compression Demo

#15

Post by blueboo » Sat, Sep 12, 2020, 9:50 AM UTC

tonic_section wrote: post #2

Hi everyone, I've been working on an implementation of a model for learnable image compression together with general support for neural image compression in PyTorch. You can try it out directly and compress your own images in Google Colab [1] or checkout the source on Github [2]. This project is based on the paper "High-Fidelity Image Compression" by Mentzer et. al. [3] - this was one of the most interesting papers I…

Huh. So a DVD at 4.7gb would go from containing 5000 5mb photos to a 700mb model + 80,000 photos.

Re: Show HN: Neural Image Compression Demo

#16

Post by tonic_section » Sat, Sep 12, 2020, 9:59 AM UTC

londons_explore wrote: post #11

garblegarble wrote: post #5

Earlier quoted context omitted.

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

I suspect if lossless reconstruction was your goal, you would want a different architecture. You would want the model to give you a conditional probability distribution for each pixel, conditioned on all previous pixels, so you could use a regular entropy coder to encode exact data.

As u/londons_explore mentioned, in theory you can train a model for lossless reconstruction - there are several papers about this, e.g. [1] is a good recent example. Lossless compressors need to learn a probability distribution over each input pixel, which amounts to maximum likelihood estimation in the original image space.

The model in the demo is a lossy compression method because it first projects the input to a lower dimensional space and performs quantization of this representation to integer values so the result can be ultimately entropy coded. It uses the mean-scale hyperprior model introduced in [1] to estimate the necessary probability distributions in the lower-dimensional space for entropy coding.

[1]: https://arxiv.org/abs/1811.12817 [2]: https://arxiv.org/abs/1802.01436

Re: Show HN: Neural Image Compression Demo

#17

Post by est31 » Sat, Sep 12, 2020, 10:04 AM UTC

godelski wrote: post #12

garblegarble wrote: post #5

Earlier quoted context omitted.

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

The model uses a GAN which does not learn the exact PDF. So not lossless, but as you can see from the images it gets extremely visually accurate results. From the README > The generator is trained to achieve realistic and not exact reconstruction. It may synthesize certain portions of a given image to remove artifacts associated with lossy compression. Therefore, in theory images which are compressed and decoded may…

> "Therefore, we emphasize that our method is not suitable for sensitive image contents, such as, e.g., storing medical images, or important documents."

As an example of this going wrong previously, xerox had once implemented compression based on deduplicating duplicate parts of documents. Obviously numbers contains tons of duplicate symbols (digits). The problem was that the scanner software deduplicated different numbers with each other, leading to wrong numbers.

http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

Re: Show HN: Neural Image Compression Demo

#18

Post by tonic_section » Sat, Sep 12, 2020, 10:07 AM UTC

godelski wrote: post #12

garblegarble wrote: post #5

Earlier quoted context omitted.

Would this work for a lossless / near lossless approach by having a final pass storing a delta between the compressed image and the original pixels, or do you think they diverge too much on a purely pixel-for-pixel basis for this to be valuable?

The model uses a GAN which does not learn the exact PDF. So not lossless, but as you can see from the images it gets extremely visually accurate results. From the README > The generator is trained to achieve realistic and not exact reconstruction. It may synthesize certain portions of a given image to remove artifacts associated with lossy compression. Therefore, in theory images which are compressed and decoded may…

Yes, the model is not lossless as this would require learning the PDF in the original input space.

However, the model does learn a conditional probability distribution over a lower-dimensional representation of the original image - this is unavoidable as entropy coding requires a distribution over discrete symbols. The GAN is almost auxiliary and not a central component of the model - in fact, you can get very good results without the GAN, but does seem to result in visually superior reconstructions.

Re: Show HN: Neural Image Compression Demo

#19

Post by 0-_-0 » Sat, Sep 12, 2020, 10:09 AM UTC

s-macke wrote: post #13

tonic_section wrote: post #10

Earlier quoted context omitted.

What was the error? I tried to make the demo notebook as robust as possible - you should be able to execute all cells in sequence once then execute cells out of sequence etc. without trouble, but it's hard to legislate for errors in Jupyter-like notebooks sometimes.

In the step # Setup model I get an error in the function call 'prepare_model' UnpicklingError: invalid load key, '<'.

I get the same

Re: Show HN: Neural Image Compression Demo

#20

Post by tonic_section » Sat, Sep 12, 2020, 10:11 AM UTC

browserface wrote: post #7

Damn I got an error at cell 19 otherwise seemed to work

GDrive doesn't download the model checkpoints correctly sometimes, leading to the following error:

``` # Setup model

I get an error in the function call 'prepare_model'

UnpicklingError: invalid load key, 'Try rerunning the download cell if you experience this - the models downloaded should be around 1.5-2GB, so if the checkpoints are 100kB in size, the download's gone wrong.