Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lepton image compression: saving 22% losslessly from images at 15MB/s

blogs.dropbox.com

Post reply on HN
Print view View original

101–110 of 196 posts

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#101

Post by ars »

daniel_rh wrote: post #28
personjerry wrote: post #27

Could someone explain to me how compression algorithms work? Shouldn't they not work very consistently, by pigeonhole principle?

It's very easy to make a JPEG file that will not compress at all. Luckily it might look like snow from a television set rather than a typical image produced by a camera. We live in a world where it is common for blue sky to occupy a portion of the frame and green grass to occupy another portion of the frame. Since images captured of our world exhibit repetition and patterns, there are opportunities for lossless compr…

> Luckily it might look like snow from a television set rather than a typical image produced by a camera.

That would actually compress rather well. A hard to compress random image would not look like TV snow (white dots, with space between them), but rather randomly colored dots that are continuous in the image.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#102

Post by cocotino »

ars wrote: post #101
daniel_rh wrote: post #28

Earlier quoted context omitted.

It's very easy to make a JPEG file that will not compress at all. Luckily it might look like snow from a television set rather than a typical image produced by a camera. We live in a world where it is common for blue sky to occupy a portion of the frame and green grass to occupy another portion of the frame. Since images captured of our world exhibit repetition and patterns, there are opportunities for lossless compr…

> Luckily it might look like snow from a television set rather than a typical image produced by a camera. That would actually compress rather well. A hard to compress random image would not look like TV snow (white dots, with space between them), but rather randomly colored dots that are continuous in the image.

Random link: "I am trying to design a cloth that, from the point of view of a camera, is very difficult to compress with JPG, resulting in big-size files (or leading to low image quality if file size is fixed)."

https://dsp.stackexchange.com/questions/2010/what-is-the-lea...

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#103

Post by kardos »

mseebach wrote: post #59
Jabbles wrote: post #49

Earlier quoted context omitted.

I don't think they're compressing this and decompressing it client side. At least I didn't get that impression from the article. You're correct ofc, download costs = 10 months of storage.

Why wouldn't they (at least, plan to)? They control the client-software.

The typical encoding/decoding rates are measured with a Xeon... my guess is that lower powered phones/netbooks will be considerably slower. I'd trade off a 20% longer sync time in exchange for not running my laptop battery down.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#104

Post by cbhl »

neuronexmachina wrote: post #32
colechristensen wrote: post #30

Earlier quoted context omitted.

The reason this can be done with JPEGs is the compression hasn't been updated. There are folks that have used h264 for compressing images and webp uses vp8 compression, both with much better results than JPEG. Lepton is cool because it helps make existing technology a whole lot better, but what we actually need is a better image format. You wouldn't see the same leap for videos because people have been working hard t…

I'd love to have something like this for archiving DVD ISOs though, where the VOBs are compressed with old-school MPEG.

Shouldn't you just use Handbrake and H.264/AAC? (Assuming your computer is fast/new enough to play it back.)

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#105

Post by mahyarm »

randyrand wrote: post #90
the_duke wrote: post #20

I really admire Dropox for open sourcing this, it shows their commitment. Saving almost a quarter of space for most images stored is something that truly gives a competitive edge. (I say most because people probably primarily have JPEG images). Especially considering how many images are probably stored on services like Dropbox. And they just gave it away to their competitors.

IMO, it shows how they don't plan to make any money from this, don't consider it a large competitive advantage, and think that the largest advantage of open sourcing it are the 'free' open source volunteers will improve the software. It's interesting that they don't consider it a large enough competitive advantage. Either that or they are using this to attract engineers.

You also open source if it's in your best interest for something to become the 'standard'.

In google there is a regret that they didn't open source a lot of stuff, because it ends up being reproduced outside of google in some form. The open source companies have an advantage in hiring and overall advancement of their product by using the open source version whatever their internal version was.

You also see it in facebook's open source initiatives, like react, buck, haxe and so on.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#106

Post by golergka »

matznerd wrote: post #69
rhaps0dy wrote: post #46

Earlier quoted context omitted.

> season 1 pretty much wiped "making the changes world a better place" from the lingo of SV companies Did it really?

Yes, the show is making the world a better place.

Better than we do.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#107

Post by daniel_rh »

mappu wrote: post #92

So it's using the VP8 arithmetic coder - how does that stack up compared to rANS?

rANS is really cool, but I think we tried it too early in the lepton research phase while some of the other ideas were still brewing... We might revisit rANS now that v1.0 is out!

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#108

Post by kardos »

mcbits wrote: post #95
sevenless wrote: post #82

I'm interested in a 'super lossy' (deep learning based?) compression. You should be able to compress movies down to a screenplay and a few stage directions. A picture should of course be worth 1000 words.

I started something like this long ago but didn't get very far with it. This was before "deep learning" and I was groping in the dark, but I think the concept is sound up to a point. The idea was to train a neural net and build up a database of features (maybe on the order of 1-10 GB, or whatever is just small enough to ship) to estimate the missing details from downscaled and extremely over-compressed JPEGs. If it w…

Sounds like the "content aware fill" algorithm [1], only for small scale features that are somehow stitched onto the low-resolution image

[1] http://www.logarithmic.net/pfh/resynthesizer

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#109

Post by therealmarv »

guelo wrote: post #73
colechristensen wrote: post #30

Earlier quoted context omitted.

The reason this can be done with JPEGs is the compression hasn't been updated. There are folks that have used h264 for compressing images and webp uses vp8 compression, both with much better results than JPEG. Lepton is cool because it helps make existing technology a whole lot better, but what we actually need is a better image format. You wouldn't see the same leap for videos because people have been working hard t…

Indeed. In one project where I converted jpegs to webp I saw around 70% savings at similar quality. This was on Android where it's fully supported. On the web pretty much only Chrome supports it, I'm not sure why Firefox, IE and Safari are hesitant.

I've heard that Safari (10) on upcoming iOS 10 supports webp. Don't know about Safari 10 on macOS Sierra.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#110

Post by nacs »

cbhl wrote: post #104
neuronexmachina wrote: post #32

Earlier quoted context omitted.

I'd love to have something like this for archiving DVD ISOs though, where the VOBs are compressed with old-school MPEG.

Shouldn't you just use Handbrake and H.264/AAC? (Assuming your computer is fast/new enough to play it back.)

The article (and the person you're replying to) is referring to lossless compression. Converting to H264 and AAC may maintain a high quality at a lower bitrate but they're definitely not lossless.
Post reply on HN