Live data from Hacker News

Lepton image compression: saving 22% losslessly from images at 15MB/s

blogs.dropbox.com

181–190 of 196 posts

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#181

Earlier quoted context omitted.

That's the idea behind the algorithm, yes. And since it's lossless, every original bit is preserved. The same idea could be applied on the Desktop client instead of on the server, which would save 22% of the bandwidth as well and make syncing faster.

Any word on when that would be implemented?

it already is.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#182

Earlier quoted context omitted.

The reason this can be done with JPEGs is the compression hasn't been updated. There are folks that have used h264 for compressing images and webp uses vp8 compression, both with much better results than JPEG. Lepton is cool because it helps make existing technology a whole lot better, but what we actually need is a better image format. You wouldn't see the same leap for videos because people have been working hard t…

I'd love to have something like this for archiving DVD ISOs though, where the VOBs are compressed with old-school MPEG.

I've always wondered if it would be possible to losslessly convert the MPEG2 DCT coefficients, motion vectors etc to the equivalent subset of h.264 and then take advantage of the better prediction and entropy encoding of the later standard.

Maybe you could even go further and actively remove keyframes (in a fully reversible way, just keep a record of where they were)

I'm not sure how much you would save, and for losslessly archiving DVDs you might be better off creating a special format like Lepton for mpeg2

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#183
post #182

Earlier quoted context omitted.

I'd love to have something like this for archiving DVD ISOs though, where the VOBs are compressed with old-school MPEG.

I've always wondered if it would be possible to losslessly convert the MPEG2 DCT coefficients, motion vectors etc to the equivalent subset of h.264 and then take advantage of the better prediction and entropy encoding of the later standard. Maybe you could even go further and actively remove keyframes (in a fully reversible way, just keep a record of where they were) I'm not sure how much you would save, and for loss…

The first part of your question was posed on Doom9 in 2009. It's the opinion of Dark Shikari, longtime lead x264 developer, that it would be possible [1].

Not sure what you mean by the keyframe removal, though. Such an act would be lossy, significantly impair any P-frames or B-frames (unless you majorly modify them), and, it frankly doesn't sound very reversible. Mind elaborating?

[1] http://forum.doom9.org/showthread.php?t=149895

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#184

Earlier quoted context omitted.

You also open source if it's in your best interest for something to become the 'standard'. In google there is a regret that they didn't open source a lot of stuff, because it ends up being reproduced outside of google in some form. The open source companies have an advantage in hiring and overall advancement of their product by using the open source version whatever their internal version was. You also see it in face…

Wait, since when is Facebook pushing for Haxe? I tried looking it up, but I'm not finding much. It would be really cool (I love Haxe). But I'm guessing you only went a bit over the top with that particular example.

Sorry that was a typo. I meant HHVM and Hack http://hhvm.com/

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#186
post #73

Earlier quoted context omitted.

Indeed. In one project where I converted jpegs to webp I saw around 70% savings at similar quality. This was on Android where it's fully supported. On the web pretty much only Chrome supports it, I'm not sure why Firefox, IE and Safari are hesitant.

I've heard that Safari (10) on upcoming iOS 10 supports webp. Don't know about Safari 10 on macOS Sierra.

Where did you read that? Since Apple has yet to commit on H.265, ( Due to Insane patent Licensing payment ), I am hoping they jump onboard with Open Media Alliance.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#188
post #92

So it's using the VP8 arithmetic coder - how does that stack up compared to rANS?

rANS is really cool, but I think we tried it too early in the lepton research phase while some of the other ideas were still brewing... We might revisit rANS now that v1.0 is out!

WebP is switching to rANS: https://chromium-review.googlesource.com/#/c/338781/

Here is a superfast implementation of rANS: https://github.com/jkbonfield/rans_static

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#189

Earlier quoted context omitted.

Thanks for testing this! For archival purposes, be sure to check the exit code of the lepton binary after compressing each JPEG: the default parameters only support a subset of JPEGs (you need to pass -allowprogressive and -memory=2048M -threadmemory=256M to support a wider variety of large or progressive JPEG files). Lepton will not write images that it is unable to compress using the settings provided. In those cas…

How much better do you think it could get with JPEGs encoded with Lepton in mind?

The best I've seen is a new compressor at http://encode.ru/threads/2459-EMMA-Context-Mixing-Compressor

Can also recompress GIF and some RAW formats, but is much, much slower

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#190
post #182

Earlier quoted context omitted.

I've always wondered if it would be possible to losslessly convert the MPEG2 DCT coefficients, motion vectors etc to the equivalent subset of h.264 and then take advantage of the better prediction and entropy encoding of the later standard. Maybe you could even go further and actively remove keyframes (in a fully reversible way, just keep a record of where they were) I'm not sure how much you would save, and for loss…

The first part of your question was posed on Doom9 in 2009. It's the opinion of Dark Shikari, longtime lead x264 developer, that it would be possible [1]. Not sure what you mean by the keyframe removal, though. Such an act would be lossy, significantly impair any P-frames or B-frames (unless you majorly modify them), and, it frankly doesn't sound very reversible. Mind elaborating? [1] http://forum.doom9.org/showthrea…

What I mean is transforming I-frames into equivalent P-frames (or even B-frames) that decode to the exact same pixels. The transformation might result in something that is bigger than a directly encoded P-frame (which can tolerate some small errors) but I suspect the new P-frame will be smaller than the original I-frame.

There is no restriction that P/B-frames only reference I-frames, so you don't even need to touch those frames.

For conversion back to MPEG2 it would be ideal to detect or mark the original I-Frames, so you can convert them back (a simpler transformation). But you could also pick any random P/B-frame and convert it to an I-frame with little issue.

Post reply on HN