Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lepton image compression: saving 22% losslessly from images at 15MB/s

blogs.dropbox.com

Post reply on HN
Print view View original

191–196 of 196 posts

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#191

Post by rleigh »

are595 wrote: post #179
jacobolus wrote: post #135

Earlier quoted context omitted.

> outperforms PNG, lossless WebP, lossless BPG, lossless JPEG2000, and lossless JPEG XR PNG was not designed to be used for photograph-like images. The rest of those were not designed to be lossless formats, the lossless version is just a tacked-on afterthought. Very unsurprising to find a codec that can beat those.

In my own tests, I found FLIF generally beats PNG (PNG Crush/Optipng both in brute-force mode) for comics (greyscale, majority white) as well, but by a less significant margin. It's also worth noting that there aren't many other lossless formats, so it's still a valid comparison. I'm sure neither TIFF nor RAW outperform FLIF either.

That's not a valid comparison for TIFF. TIFF is a container which supports multiple compression algorithms. You could plug FLIF in as a compression algorithm and use it directly.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#192

Post by rleigh »

jszymborski wrote: post #79
SapphireSun wrote: post #66

This is amazing, we've been struggling with JPG storage and fast delivery at my lab (terabytes and petabytes of microscopy images). We'll be running tests and giving this a shot!

jpg is a weird format to be storing microscopy images, no? Usually end up in some sort of bitmap TIFF (or their Zeiss/etc. proprietary format) from what I've seen.

Some TIFF-derived container is the most common representation. But note even these could use JPEG/J2K if desired.

Most microscopy images are stored uncompressed or with lossless compression. But unfortunately this doesn't scale with newer imaging modalities. Here's two examples:

Digital histopathology. Whole-slide scanners can create huge images e.g. 200000x200000 and larger. These are stored using e.g. JPEG or J2K in a tiled BigTIFF container, with multiple resolution levels. Or JPEG-XR. When each image is multiple gigabytes, lossless compression doesn't scale.

SPIM involves imaging a 3D volume by rotating the sample and imaging it from multiple angles and directions. The raw data can be multiple terabytes per image and is both sparse and full of redundant information. The viewable post-processed 3D image volume is vastly smaller, but also still sparse.

For more standard images such as confocal or brightfield or epifluorescence CCD, lossless storage is certainly the norm. You don't really want to perform precise quantitative measurements with poor quality data.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#193

Post by rleigh »

halomru wrote: post #77
sliverstorm wrote: post #70

Earlier quoted context omitted.

Is JPEG, PNG, etc a container format that could use arbitrary compression, like MKV? Or do we need a brand new format.

PNG is a very nice, extensible container format. In principle you can use it to store arbitrary image data, and still make use of all established metadata fields. In practice you probably shouldn't do that without changing the file extension and the magic bytes to avoid confusion among users and poorly written software.

While PNG (and libpng) are reasonable for their intended purpose as a GIF replacement for simple RGB and greyscale graphics, it doesn't compare when it comes to the power and flexibility of TIFF, from sample format and depth, to sample count, image orientation, tiling and many additional features.

TIFF is showing its age at this point, with a some high-end scientific applications switching to HDF5. But PNG simply isn't featureful enough for this type of data.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#194

Post by are595 »

rleigh wrote: post #191
are595 wrote: post #179

Earlier quoted context omitted.

In my own tests, I found FLIF generally beats PNG (PNG Crush/Optipng both in brute-force mode) for comics (greyscale, majority white) as well, but by a less significant margin. It's also worth noting that there aren't many other lossless formats, so it's still a valid comparison. I'm sure neither TIFF nor RAW outperform FLIF either.

That's not a valid comparison for TIFF. TIFF is a container which supports multiple compression algorithms. You could plug FLIF in as a compression algorithm and use it directly.

Alright, well against current commonly supported TIFF compression algorithms such as LZW, Packbits, and ZIP.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#195

Post by ptspts »

Some statistics. TL;DR Lepton indeed provides about 22% of size savings.

When running Lepton through JPEG photos downloaded from Flickr, I've got about 23.37% of size savings.

When running Lepton through JPEGs generated by mozjpeg (default settings: -quality 75, progressive) from JPEG photos download from Flickr, I've got 22.63% of size savings.

The mozjpeg output is about 3.83 times smaller than the original JPEG photo, on average.

Re: Lepton image compression: saving 22% losslessly from images at 15MB/s

#196

Post by nogridbag »

ptspts wrote: post #169
nogridbag wrote: post #98

Does anyone know how this compares to jpegoptim? https://github.com/tjko/jpegoptim I'm hitting the limits in my OneDrive account and jpegoptim seemed to reduce my photos quite a bit.

jpegoptim has lossless and lossy modes. In lossless mode it preserves all pixels, but it doesn't preserve the file itself. Lossless jpegoptim is comparable to Lepton. In general, Lepton can tends to give better improvements, because it uses a different output file format. How much better depends on your input files, you should try both. I'd say 22% for Lepton and 5% for jpegoptim, based on fading past memories of min…

Thanks. I posted that before trying out Lepton not realizing the artifact is no longer a jpg.
Post reply on HN