Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HEIF – High Efficiency Image File Format

nokiatech.github.io

Post reply on HN
Print view View original

31–40 of 122 posts

Re: HEIF – High Efficiency Image File Format

#31

Post by derefr »

masklinn wrote: post #18
amelius wrote: post #16

Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? I mean, looking at WASM, sandbox technology is sufficiently strong. And performance and availability aren't really an issue either, because for specific cases we can fall back to what we are doing now.

> Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? Image formats are already some of the biggest attack surface of modern systems, "executable" image formats (hello PDF) have an absolutely ghastly track record there. And not being able to open an image if you don't have an internet connection sounds dreadful.

Re: attack surface—PDF is complex because it requires a ton of interaction (you can fill PDF forms, etc.) Image decoders should just be pure functions—binary stream in, matrix of pixel structs out. Easily sandboxed—you shouldn't need access to any system calls while doing that decoding.

Re: requiring an Internet connection—you still do require an Internet connection to display an image, if it's in a format whose decoder you don't currently have installed; you just currently have to manually 1. figure out what the format even is, 2. select a library package for a decoder for that format, and 3. install that package.

Presumably such libraries would be able to register decoder-URLs they are (non-reference) implementations of—like they register MIME types today—so your system would be able to display standard formats no problem; it'd just be the weird rare or new ones that would trigger the zero-install process for a decoder.

Re: HEIF – High Efficiency Image File Format

#32

Post by sprash »

agentgt wrote: post #17
Adamantcheese wrote: post #12

I wonder how it compares to FLIF, which looks like it has a formal release finally. http://flif.info/

It looks like FLIF doesn't really support lossy well (its sort of fake lossy). "FLIF does not have a lossy mode, but interlaced files can be decoded progressively so we can simply use something like dd if=lossless.flif of=lossy.flif bs=1024 count=5" You can see this in the examples where BPG does well in stream loading: https://nokiatech.github.io/heif/technical.html HEIF would probably be similar to BPG in performan…

> It looks like FLIF doesn't really support lossy well

I don't know what you mean by well. As far as I'm concerned lossy FLIF beats JPEG easily. Here is a comparision:

http://flif.info/lossy-artifacts.html

Re: HEIF – High Efficiency Image File Format

#33

Post by amelius »

masklinn wrote: post #18
amelius wrote: post #16

Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? I mean, looking at WASM, sandbox technology is sufficiently strong. And performance and availability aren't really an issue either, because for specific cases we can fall back to what we are doing now.

> Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? Image formats are already some of the biggest attack surface of modern systems, "executable" image formats (hello PDF) have an absolutely ghastly track record there. And not being able to open an image if you don't have an internet connection sounds dreadful.

But, as I already implied, if we can safely run arbitrary "binary" code of the internet (see WASM), why can't we run arbitrary code of a decoder?

Re: HEIF – High Efficiency Image File Format

#34

Post by jacobolus »

zerocrates wrote: post #21
niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

It mentions that it supports not only sequences but inter-frame prediction (so P- and possibly B- frames I assume), which is sort of surprising to me but of course makes sense especially given Apple's "image burst" usage. I'm also suprised to learn that WebP apparently supports P-frames in its animations. What's really different between this and the existing usages of the MP4 container for images, though? Is it just…

https://nokiatech.github.io/heif/technical.html

Re: HEIF – High Efficiency Image File Format

#36

Post by [deleted] »

wereHamster wrote: post #29
niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

At last for the web (which is what WebP targets mostly, right?) you don't need broad support. We have the new element where the UA selects the most appropriate image from a list of options. So Chrome will prefer webp, other browsers will pick jpeg, png or whatever else they support at the moment.

[deleted]

Re: HEIF – High Efficiency Image File Format

#37

Post by jacobolus »

wereHamster wrote: post #29
niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

At last for the web (which is what WebP targets mostly, right?) you don't need broad support. We have the new element where the UA selects the most appropriate image from a list of options. So Chrome will prefer webp, other browsers will pick jpeg, png or whatever else they support at the moment.

If possible, I’d much prefer having a single format that is widely supported, hardware accelerated, and with a state-of-the-art codec. Making content authors create multiple different files in different formats—some with deficient feature support—sounds like a pain in the butt for everyone.

If we can get everyone to agree on a format that handles layers, exposure bracketing, transparency, animation, high bit depth, different color models, both lossy and lossless compression, every common type of metadata, etc., that would be great.

Re: HEIF – High Efficiency Image File Format

#38

Post by TD-Linux »

zokier wrote: post #26

While the basic idea of using HVEC for still image compression is good, I'm somewhat concerned on how complex the whole format, especially the container, for HEIF is. It is kinda cute and from one viewpoint pragmatic to reuse existing container format, but I'm afraid the flexibility might have unforeseen and undesirable side-effects. And of course there is the problem of different implementations supporting different…

They've already specified three different codec mappings - JPEG, AVC, and HEVC - so to open all HEIF files, you presumably need to implement all of the decoders.

Worse, there is a financial incentive to implement only a subset of the decoders - HEVC is more expensive than AVC which is more expensive than JPEG. Even things like 4:4:4 color are more expensive than 4:2:0 color for HEVC.

Re: HEIF – High Efficiency Image File Format

#39

Post by zokier »

niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

I think the significance of compatibility is increasingly diminishing due the effect of "clouds" and other walled gardens that handle your data in a more integrated manner. With the advent of wasm etc it is not completely unreasonable to ship your own image decoder with the application, and optionally a JPEG encoder for on-demand client side re-encoding for exporting images.

There is just less expectations to be able to interoperate via "dumb files" in the "app ecosystem" where the apps run in their own isolated sandboxes and import/export are explicit actions.

Not saying that this is the best thing, but the situation is different enough from the time when WebP/WMP were introduced that HEIF actually might have a fighting chance. The fact that HEIF appears to be far bigger improvement over JPEG than previous efforts of course also helps.

Re: HEIF – High Efficiency Image File Format

#40

Post by pornel »

It makes a lot of sense for Apple to use H.265 as a base for the format. It's much more efficient than all the other JPEG-killers (easily beats WebP by a wide margin). Apple already pays for the patents, and has invested in H.265 hardware and software for H.265 video.

However, the HEIF wrapper for H.265 strikes me as quite complex. It brings baggage of ISO wrapper formats and tries to support everything ever crammed into any image-ish format. That, in addition to patent licensing, may be another difficulty in widespread support for the format. It will be hard to implement robust, secure and fully-featured players.

Post reply on HN