What’s the best lossless image format? • Hacker News Board

Re: What’s the best lossless image format?

#61

Post by bob1029 » Tue, Jun 07, 2022, 9:48 PM UTC

petesergeant wrote: post #25

ptidhomme wrote: post #6

I think decompression times would be a relevant criteria, but they're surprisingly not there...

Quite, and encoding speed, which is included, seems almost irrelevant

Encoding speed can be very important, especially if you are seeking to decode the resulting image as quickly as possible at all times. Certain image formats are wildly better real-time containers than others. Encoding speed is irrelevant unless it's very low or very high. Most people are thinking amortize the decode against the encode and don't even consider what would happen if you took it into the other extreme.

Did you know that, with the correct SIMD code, you can encode a 1080p bitmap into a 4:4:4 JPEG within 10 milliseconds on a typical consumer PC? Encoding times like this open up an entire dimension of new possibilities.

I do have a side project where I use libjpegturbo to synchronize a frame buffer between server & web client (i.e. 60 FPS real-time). HN may frown on using [m]jpeg for real-time applications today, but in my experience it has proven to be a very robust path. Bandwidth is really the only tradeoff (and it is a big one). This is the very last thing Netflix would want to do (short of sending RAW files to your TV), but it does provide some very compelling attributes in other areas such as streaming gaming.

Our networks are only getting faster over time. Intraframe video compression techniques are really elegant in their simplicity and resilience to motion-induced artifacts (as seen with x264, et. al.). They also can provide lower latency and easier recovery for clients (since every frame is a keyframe).

For real-time applications, I'd happily trade some extra bits per second to have a final result sooner. Especially if it looked better.

Re: What’s the best lossless image format?

#62

Post by throw10920 » Tue, Jun 07, 2022, 10:04 PM UTC

pmoriarty wrote: post #43

petesergeant wrote: post #25

Earlier quoted context omitted.

Quite, and encoding speed, which is included, seems almost irrelevant

Why is encoding speed irrelevant? I'm currently in the process of scanning and archiving lots of photographs as high-resolution 16-bit PNG's and encoding speed is definitely a significant influence on how long this process takes.

You missed the "almost".

The vast majority of use-cases and users barely care about encoding performance. Your use is in the tiny minority. That isn't to say that you don't have a use-case, just that because of how rarely encoding speed matters, it truly is "almost irrelevant" for 99.99% of users 99.99% of the time.

--------------

It definitely would be nice if JPEG-XL also had good encoding performance, though. There's an experimental fast encoder in development currently[1], although before that lands, it would be nice if your scanning tool supported a workflow somewhat like "scan in parallel with image encoding, present encoded images to user, and allow them to asynchronously go back and re-scan/encode/review pages that came out badly".

[1] https://github.com/libjxl/libjxl/tree/main/experimental/fast...

Re: What’s the best lossless image format?

#63

Post by xigoi » Tue, Jun 07, 2022, 10:07 PM UTC

mg wrote: post #8

I would love to see an estimation of how small real world images would get if we could calculate their Kolmogorov complexity. Aka if we could find the shortest program that produces the image as output. Some images obviously would become very small. For example an image that is all blue, a spiral, the flag of the United States, the Mandelbrot set ... But what about the average portrait for example?

I wonder how much could be achieved just using a vector format with basic programming constructs like for loops.

Re: What’s the best lossless image format?

#64

Post by andbberger » Tue, Jun 07, 2022, 10:09 PM UTC

Iv wrote: post #48

lajamerr wrote: post #22

JPEG XL is the standard to beat them all. Like how Opus became the definitive codec for lossy compression for the web. JPEG XL covers nearly every use case for images, even replacing GIFs with it's animation support. Existing JPEGs can also be re-encoded losslessly into JPEG XL format/data structure and get significant file size savings. This backwards compatiability is very important in transitioning to web to JPEG…

How does it relate to JPEG2000? I may soon have to recommend a lossy but potentially lossless image format for 3D medical images (with 16 bits channels), would JPEG XL fit this bill?

you should use NIfTI, which is the de facto standard for 3D medical imaging.

not compressed, but handles the voxel-image space transform which is far far more important and quite complicated.

you can slap gzip on top of it and most libraries/FIJI will transparently decompress

Re: What’s the best lossless image format?

#65

Post by Dylan16807 » Tue, Jun 07, 2022, 10:22 PM UTC

pmoriarty wrote: post #43

petesergeant wrote: post #25

Earlier quoted context omitted.

Quite, and encoding speed, which is included, seems almost irrelevant

Why is encoding speed irrelevant? I'm currently in the process of scanning and archiving lots of photographs as high-resolution 16-bit PNG's and encoding speed is definitely a significant influence on how long this process takes.

How many megabytes per second is your scanner feeding you? I'm very surprised that encoding a PNG is slower than scanning.

Re: What’s the best lossless image format?

#66

Post by detuur » Tue, Jun 07, 2022, 10:23 PM UTC

Iv wrote: post #59

lajamerr wrote: post #50

Earlier quoted context omitted.

The future isn't here yet unfortunately. I imagine the doctors would view these images in multitudes of ways(browsers, operating systems, image viewers) with varying level of support. So encoding in a standard that is yet to be entirely finalized and adopted is a bad move as of now. You'd be best off with going with JPEG2000 still if that's what you prefer or DICOM.

Oh adoption does not have to be that good. It is for researchers right now, to exchange results. So the only requirement is that it is possible to use it with a somewhat mature lib in python.

I absolutely recommend JXL. It is developed by the JPEG Group as a successor to JPEG2000 and performs better in nearly every metric (big exception currently being encoder speed). JPEG2000 isn't really widely supported, but does have many more libraries available.

So unless you expect library availability and/or encoder speed to hurt adoption, go for JXL.

Re: What’s the best lossless image format?

#67

Post by pmoriarty » Tue, Jun 07, 2022, 10:24 PM UTC

Dylan16807 wrote: post #65

pmoriarty wrote: post #43

Earlier quoted context omitted.

Why is encoding speed irrelevant? I'm currently in the process of scanning and archiving lots of photographs as high-resolution 16-bit PNG's and encoding speed is definitely a significant influence on how long this process takes.

How many megabytes per second is your scanner feeding you? I'm very surprised that encoding a PNG is slower than scanning.

It's not slower than scanning, but I still spend a significant amount of time waiting for the maximum compression PNG encoding to complete.

Re: What’s the best lossless image format?

#68

Post by l33t2328 » Tue, Jun 07, 2022, 10:26 PM UTC

SR2Z wrote: post #26

mg wrote: post #8

I would love to see an estimation of how small real world images would get if we could calculate their Kolmogorov complexity. Aka if we could find the shortest program that produces the image as output. Some images obviously would become very small. For example an image that is all blue, a spiral, the flag of the United States, the Mandelbrot set ... But what about the average portrait for example?

You cannot calculate Kolmogorov complexity in all cases because that would involve solving the halting problem. It is worth pointing out that certain formats like AVIF do sort of do this: they use a few different algorithms, see which combination of algo/block split removes the most entropy, and then compress _that_. In practice, this is pretty good for most non-noisy images.

Wouldn’t you want entropy to increase when you compress?

Re: What’s the best lossless image format?

#69

Post by detuur » Tue, Jun 07, 2022, 10:27 PM UTC

Melatonic wrote: post #49

If you want something that is going to be around for awhile and has a LOT of features you probably do not need then you are going to want to go probably with a .DPX or .EXR which are the standard for the VFX industry as well as film in general. On the consumer side I guess adobe .DNG might not be bad either or for maximum compatibility but then maybe a Tiff. For video I personally like Cineform but both ProRes and DN…

How do EXR and JXL compare to each other, feature-wise?

Re: What’s the best lossless image format?

#70

Post by aaaaaaaaaaab » Tue, Jun 07, 2022, 10:47 PM UTC

I want to see if they’re really lossless. I know for a fact that PNG is. But last I checked HEIF didn’t support lossless RGB images, only lossless YUV… So HEIF was unusable for bit-perfect reproduction of the original.