Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulkan video extensions for accelerated H.264 and H.265 encode

khronos.org

Post reply on HN
Print view View original

71–80 of 110 posts

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#71

Post by geraldhh »

kllrnohj wrote: post #12
shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

> Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos

they are also plentiful, vendor specific and unstable.

a generic solution would be well appreciated

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#72

Post by pjc50 »

0x1ceb00da wrote: post #33
monocasa wrote: post #31

Earlier quoted context omitted.

Interestingly, quite a bit of video decode isn't as parallelizable as you might think and isn't a great fit for GPUs. For instance the initial Huffman decoding of the stream is essentially an intrinsically sequential process.

Is there a reason we don't have parallelizable video formats?

What does that mean? A video is inherently a series of frames. And using that sequential nature is critical to achieving good compression.

However, most formats allow for some parallelism at the "macroblock" level - you can usually decode all 16x16 pixels simultaneously. To some extent you can decode macroblocks separately, but "intra prediction" requires you to have the ones above and to the left available.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#73

Post by _flux »

pjc50 wrote: post #72
0x1ceb00da wrote: post #33

Earlier quoted context omitted.

Is there a reason we don't have parallelizable video formats?

What does that mean? A video is inherently a series of frames. And using that sequential nature is critical to achieving good compression. However, most formats allow for some parallelism at the "macroblock" level - you can usually decode all 16x16 pixels simultaneously. To some extent you can decode macroblocks separately, but "intra prediction" requires you to have the ones above and to the left available.

It means that e.g. in H265 you can split the frame into tiles that are much larger than macroblocks: it's like the video is a set of n videos that are then composed to build a complete frame. This allows both encoding and decoding to work in parallel even inside a single frame. In addition, in VR 360 context, it allows decoding only frames the viewer can see.

It reduces the compression efficiency, but only a little, because the tiles will be quite large anyway, such as 6 tiles per frame (but can be more for 360 video applications). It can be limited by the number of decoding instances a piece of hardware can have concurrently.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#74

Post by pjc50 »

_flux wrote: post #73
pjc50 wrote: post #72

Earlier quoted context omitted.

What does that mean? A video is inherently a series of frames. And using that sequential nature is critical to achieving good compression. However, most formats allow for some parallelism at the "macroblock" level - you can usually decode all 16x16 pixels simultaneously. To some extent you can decode macroblocks separately, but "intra prediction" requires you to have the ones above and to the left available.

It means that e.g. in H265 you can split the frame into tiles that are much larger than macroblocks: it's like the video is a set of n videos that are then composed to build a complete frame. This allows both encoding and decoding to work in parallel even inside a single frame. In addition, in VR 360 context, it allows decoding only frames the viewer can see. It reduces the compression efficiency, but only a little,…

So the answer to the original comment "Is there a reason we don't have parallelizable video formats?" is "we do have them".

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#75

Post by magicalhippo »

geraldhh wrote: post #70
magicalhippo wrote: post #50

Earlier quoted context omitted.

Also, to get best compression you don't want to do key frames at even intervals, but rather when it pays off (ie large changes between two frames, like a scene change).

seems rather obvious and i would imagine most encoding algos act accordingly

The point was rather that it makes parallel encoding more difficult. With a fixed interval it is trivial, you just split the incoming stream based on the fixed rate, and let each thread work on separate intervals.

With adaptive key frame placement you don't know the intervals up front, and they might have wildly different lengths. Some might be hundreds of frames, some might be just a few.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#77

Post by luthMan »

shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

What about systems where there is now hardware support for AV1? How would Vulkan implement that?

Also, maybe someone knows more about this, but I thought that most software like Firefox, MPV and VLC are using dav1d for AV1 decoding as it seems to be the most performant AV1 decoder (in Software) out there at the moment.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#78

Post by bmicraft »

pjc50 wrote: post #74
_flux wrote: post #73

Earlier quoted context omitted.

It means that e.g. in H265 you can split the frame into tiles that are much larger than macroblocks: it's like the video is a set of n videos that are then composed to build a complete frame. This allows both encoding and decoding to work in parallel even inside a single frame. In addition, in VR 360 context, it allows decoding only frames the viewer can see. It reduces the compression efficiency, but only a little,…

So the answer to the original comment "Is there a reason we don't have parallelizable video formats?" is "we do have them".

The answer seems to be "parallel videos can be decoded in parallel"

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#79

Post by jdiff »

mattst88 wrote: post #19
jdiff wrote: post #17

Earlier quoted context omitted.

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

> Firefox still has no support for hardware video decode on Linux. That's not true at all. See: https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

My bad, that's what I get for commenting before bed. I meant to tack on an "on Nvidia" in there. AMD and Intel are well supported with VAAPI, but I've never managed to get any of the NV-to-VAAPI wrappers and shims working.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#80

Post by bmicraft »

pjmlp wrote: post #67
shmerl wrote: post #58

Earlier quoted context omitted.

Those who use it appreciate their efforts. You aren't using it, why are you even complaining especially with complete nonsense comments. Anti Linux shilling should be getting old.

Unfortunely I still have to, from time to time. Luckly Android, ChromeOS and WebOS as proper Linux distributions have replaced most of it.

Android hasn't displaced Linux on phones, there wasn't any.

ChromeOS hasn't displaced Linux on school laptops, there wasn't any.

LG WebOS hasn't displaced "Linux", it competes with Google TV (formerly Android TV)

Post reply on HN