Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HEIF – High Efficiency Image File Format

nokiatech.github.io

Post reply on HN
Print view View original

101–110 of 122 posts

Re: HEIF – High Efficiency Image File Format

#101

Post by lsiebert »

zokier wrote: post #26

While the basic idea of using HVEC for still image compression is good, I'm somewhat concerned on how complex the whole format, especially the container, for HEIF is. It is kinda cute and from one viewpoint pragmatic to reuse existing container format, but I'm afraid the flexibility might have unforeseen and undesirable side-effects. And of course there is the problem of different implementations supporting different…

like usb-c and thunderbolt compatibility, I am worried there will be a lot of confusion and what works with what. Less in the area of devices that attach to your hw direction, but more when it comes to consumer devices with HW decoder support and getting these formats to display.

Re: HEIF – High Efficiency Image File Format

#102

Post by jpap »

zerocrates wrote: post #21
niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

It mentions that it supports not only sequences but inter-frame prediction (so P- and possibly B- frames I assume), which is sort of surprising to me but of course makes sense especially given Apple's "image burst" usage. I'm also suprised to learn that WebP apparently supports P-frames in its animations. What's really different between this and the existing usages of the MP4 container for images, though? Is it just…

HEIF is actually very closely related to MP4 (or really, the ISO Base Media File Format (BMFF); and historically, QuickTime which is what they are both based on).

You are right -- the main difference is that BMFF was originally designed for time-based media (video, audio) but isn't in itself well suited towards media that isn't time based, like images and their associated metadata (e.g. Exif).

HEIF extends the ISO BMFF so that you can have untimed media -- one or more photos, a collection -- or even mixed media comprising both untimed and timed media. Apple's live photo is a good example of mixed media, comprising a photo and a video.

But the HEIF container format goes so much further. You could have image sequences -- for example a higher-quality version of Animated GIF (using I-frames only, or P/B for more compression) with looping -- tiling, auxiliary images like depth maps and alpha channels, or even stereo images with a left and right channel.

Though originally HEIF came out of the HEVC standards track, hence the words "High Efficiency" in the name, it was later extended to include other codecs like JPEG, AVC. There's no reason it couldn't be extended to include VP9, PNG, or any other codec.

Think of HEIF as a versatile, extensible, standardized container format. The media coding scheme is separable. This is a big deal for the future of image/media coding because we're no-longer locked into "yet another format" that's tied to the codec. With the ISO BMFF box model, HEIF can grow to adapt new constructs and codecs for some generations to come.

Re: HEIF – High Efficiency Image File Format

#103

Post by jpap »

jacobolus wrote: post #37
wereHamster wrote: post #29

Earlier quoted context omitted.

At last for the web (which is what WebP targets mostly, right?) you don't need broad support. We have the new element where the UA selects the most appropriate image from a list of options. So Chrome will prefer webp, other browsers will pick jpeg, png or whatever else they support at the moment.

If possible, I’d much prefer having a single format that is widely supported, hardware accelerated, and with a state-of-the-art codec. Making content authors create multiple different files in different formats—some with deficient feature support—sounds like a pain in the butt for everyone. If we can get everyone to agree on a format that handles layers, exposure bracketing, transparency, animation, high bit depth, d…

Yes! This is one of the motivations behind HEIF: it's codec agnostic and extensible. You can use HEIF with JPEG and AVC for example, or some new codec down the line.

It's also extensible, in that you can define new item types and box structures to describe any new imaging construct you like. For example, you could host images with alpha, depth maps, stereo left/right channels, a HDR image with each of the raw brackets and the final fused image, etc.

The format relies upon the ISO Base Media File Format file-type branding to distinguish the infinite possibilities into a clear set of capabilities required for playback; in the same way MPEG4 does for video codec profile and levels.

Re: HEIF – High Efficiency Image File Format

#104

Post by emn13 »

pornel wrote: post #60
throwasehasdwi wrote: post #58

Nokia owns several HEVC patents, in fact I think they're currently suing apple about it. This is a veiled attempt to get everyone to use their patented tech so we can relive the mp3/mp4 clusterfuck. You would be a fool to use this for anything. What's so bad about WebP? It's not the best thing in the universe but it beats JPEG and PNG and has a wide support base. With a shim it has native support in most browsers any…

> What's so bad about WebP? WebP is based on VP8, which is now obsolete. It was meant to be a H.264 competitor, a generation behind H.265 and VP9/VP10. WebP compression efficiency is much closer to JPEG than H.265. This format beats WebP by a margin wider than WebP was smaller than JPEG.

And people today still use jpeg routinely, so I'm not sure any of that is really a killing blow.

Re: HEIF – High Efficiency Image File Format

#105

Post by jpap »

pornel wrote: post #40

It makes a lot of sense for Apple to use H.265 as a base for the format. It's much more efficient than all the other JPEG-killers (easily beats WebP by a wide margin). Apple already pays for the patents, and has invested in H.265 hardware and software for H.265 video. However, the HEIF wrapper for H.265 strikes me as quite complex. It brings baggage of ISO wrapper formats and tries to support everything ever crammed…

While the full standard text and machinery can get quite complex, you can still construct relatively simple files housing a single image, thumbnail, and Exif that isn't that much more complex than a typical JPEG file. Compared to JPEG, where you need to define quantization tables, Huffman tables, etc. in marker segments, much of the codec-related complexity in HEIF is layered within the compressed image payload (e.g. HEVC NAL units).

Besides the coding efficiency, JPEG really hit the big time because of the readily available libjpeg cross-platform implementation. In this case, HEIF can leverage existing implementations of the ISO Base Media File Format box model; it builds heavily on that standard.

Nokia is doing the right thing by releasing their format handling library, though perhaps they could loosen up their license to include commercial use. ;-) Hopefully there are some developers amongst us inspired enough to start a new open project that, like libjpeg, brings HEIF to the masses.

Re: HEIF – High Efficiency Image File Format

#106

Post by emn13 »

agentgt wrote: post #81
sprash wrote: post #32

Earlier quoted context omitted.

> It looks like FLIF doesn't really support lossy well I don't know what you mean by well. As far as I'm concerned lossy FLIF beats JPEG easily. Here is a comparision: http://flif.info/lossy-artifacts.html

Whoops looks like I pasted the wrong link. Here is the correct link: http://flif.info/example.html And yes "well" is relative and what I meant by that is that its not readily easy for a normal user to do it. BTW I think Flif is damn good particularly progressive decoding. Its unclear how good HEIF progressive decoding is.

Yeah, even on that fair; jpg-unfriendly example jpeg still clearly beats flif above 22kb. And below 22kb - sure, FLIF is less bad, but it's still unusable. Also, the examples below 22kb are a little unfair to most codecs in practice, since you'd get far better results by downscaling first.

Re: HEIF – High Efficiency Image File Format

#107

Post by jpap »

amelius wrote: post #16

Why don't we store a url to the decoder inside images and other compressed files for maximum flexibility? I mean, looking at WASM, sandbox technology is sufficiently strong. And performance and availability aren't really an issue either, because for specific cases we can fall back to what we are doing now.

That's actually not a bad idea!

The HEIF examples on the Nokia HEIF site do something similar; they implement a HEVC decoder (and do the HEIF container processing) all in JavaScript using http://www.libde265.org. The images you see are decoded into a HTML5 canvas.

Re: HEIF – High Efficiency Image File Format

#108

Post by jpap »

niftich wrote: post #25
tehabe wrote: post #13

So HEIF is for HEVC what WebP is for VP9.

Roughly, BPG [1] is for HEVC as what WebP is for VP8 or VP9, in the sense that you take an I-frame out of a video format and stuff it into an established, classic image container. Compared to this, HEIF is a different way of packaging HEVC frames; namely into ISOBMFF (MPEG-4 Part 12), the most basic level of MPEG container. This comes with benefits [2] that come along with using that particular container, but also dr…

> I'd be curious if the two "representations" are losslessly convertible, for example.

They are: you could just extract the HEVC NAL units, and re-write them into a MP4 or QuickTime container, making sure to properly place the codec configuration box, etc.

HEIF also goes beyond a sequence of frames, in that it can describe alpha planes, depth maps, tiling, etc. In that case there might not be an analog with a standard video. If you really wanted to decompose a HEIF container, you might choose to extract the raw media into elementary streams (for HEVC or AVC; or if you're using the JPEG codec, just plain JPEG files) adjacent to any metadata like Exif, etc. This is essentially what Nokia's conformance files are [1].

[1] https://github.com/nokiatech/heif_conformance/tree/master/bi...

Re: HEIF – High Efficiency Image File Format

#109

Post by jpap »

niftich wrote: post #20

This seems conceptually most similar to Fabrice Bellard's proposal 'BPG' [1], which was essentially a lightweight wrapper around an HEVC I-frame. The format got small amounts of attention in encoding circles, but didn't pick up mainstream traction. The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.),…

> This format doesn't really blaze new ground (EDIT: it does in the sense that it defines a container format to express image-y constructs like still images and sequences of images in an ISO media container, but see my other comment that asks how this is similar but different to video [3]), but if this repackaging and the resulting code donation and political clout helps it gain traction, we still would gain a lot.

You need to think about HEIF as the container and disconnected from the codec used to compress the media content. In that sense, HEIF is a massive trailblazer. It gives us an ISO standard to describe image and mixed media consistent with existing methods (i.e. MP4) while allowing for new constructs (i.e. useful features of the future in addition to tiling, depth map, stereo, etc.) and new codecs as they are developed and adopted. HEIF is a universal forward-looking format.

Until now, almost all of the major imaging formats were tied to the codec and not terribly extensible to new use cases. While BPG is a great format in the short term (it gave us HEVC coded images around 2.5 years ago), it isn't an ideal choice for the long term when viewed as above.

Post reply on HN