Live data from Hacker News

Vulkan video extensions for accelerated H.264 and H.265 encode

khronos.org

31–40 of 110 posts

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#31

So to clarify, this still leverages the fixed function hardware video encoders present alongside many GPUs from eg. Intel/NVIDIA/AMD, and would not (for example) be possible to use on just a GPU itself (without said encoding hardware)?

Interestingly, quite a bit of video decode isn't as parallelizable as you might think and isn't a great fit for GPUs. For instance the initial Huffman decoding of the stream is essentially an intrinsically sequential process.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#32
post #30

Earlier quoted context omitted.

> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA. Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers onl…

You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then? Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like t…

Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#33

So to clarify, this still leverages the fixed function hardware video encoders present alongside many GPUs from eg. Intel/NVIDIA/AMD, and would not (for example) be possible to use on just a GPU itself (without said encoding hardware)?

Interestingly, quite a bit of video decode isn't as parallelizable as you might think and isn't a great fit for GPUs. For instance the initial Huffman decoding of the stream is essentially an intrinsically sequential process.

Is there a reason we don't have parallelizable video formats?

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#34

Earlier quoted context omitted.

Interestingly, quite a bit of video decode isn't as parallelizable as you might think and isn't a great fit for GPUs. For instance the initial Huffman decoding of the stream is essentially an intrinsically sequential process.

Is there a reason we don't have parallelizable video formats?

Largely because a lot of compression is contextual to the data around it. That makes most decompression a poor fit for parallelism.

The primary choices are space savings and power efficiency. Dedicated hardware and serial decoding/encoding often win out as a result.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#35
post #30

Earlier quoted context omitted.

You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then? Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like t…

Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.

[deleted]

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#36
post #30

Earlier quoted context omitted.

You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then? Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like t…

Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.

What of it? So does nvidia-vaapi-driver. They're third party projects, that's very different from "supported by the vendor", and doesn't change the fact that NVIDIA as a vendor offers no support for VA-API.

By the way, nouveau's support is currently limited and not useful: https://nouveau.freedesktop.org/VideoAcceleration.html see Video engine support status table, only old GPUs and no H.265 or AV1 support.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#37

Earlier quoted context omitted.

Interestingly, quite a bit of video decode isn't as parallelizable as you might think and isn't a great fit for GPUs. For instance the initial Huffman decoding of the stream is essentially an intrinsically sequential process.

Is there a reason we don't have parallelizable video formats?

Modern video codecs use motion prediction algorithms to encode the information between key frames which depends on the entire frame and the ones immediately preceding it.

You can encode the independent groups of pictures (the key frame and all the subsequent predictive frames until the next key frame) in parallel but at least with 4k video you hit memory bandwidth limitations quite quickly.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#38
post #21

Earlier quoted context omitted.

> Firefox still has no support for hardware video decode on Linux. That's not true at all. See: https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

How can I confirm whether Firefox is actually using hardware decode for videos on my machine for a given codec? Is there a magic keyword I should search for within the about:support page, or is it recorded somewhere else? Mine shows for instance "VP9_HW_DECODE default available", does this means it's using hardware decode for VP9, or merely that it might be possible for my Firefox version?

On an Intel GPU, you can check the 'Video' percentage in `sudo intel_gpu_top` (from the igt-gpu-tools package) while playing a video.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#39
post #17

Earlier quoted context omitted.

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

> Firefox still has no support for hardware video decode on Linux. That's not true at all. See: https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

Whilst Firefox may support hardware video decoding, Mesa since March 2022 disables patent encumbered codecs by default[1], and distributions such as Fedora and OpenSuse do not explicitly enable these patent encumbered codecs to avoid possible legal problems. Even Gentoo (built from source code by the user) requires the user to explicitly enable a USE flag (proprietary-codes) to use patent encumbered codecs.[2]

The thought process is that AMD, NVIDIA, Intel and the likes are not providing a patent license with their hardware.[3] They are instead just supplying part of an overall system that together with operating system kernel, display manager software, video player software, etc allows the decoding and encoding of patent encumbered video files. Open source software projects and distributions are concerned they'd be found to be infringing patents by enabling a complete solution out-of-the-box. Hence they put some hurdles in place so that a user has to go out of their way to separately piece together the various parts to form a complete system capable of encoding and decoding patent encumbered codecs.

edit: To clarify, if your Intel or AMD GPU/APU supports a patent-free codec such as AV1 (most GPUs/APUs available for sale?), Firefox on a standard Linux distribution will use hardware video decoding out of the box by default for the patent-free codec. The issue is really one of whether you're sourcing content from a provider that uses a good choice of codec like AV1. The good news is that patent trolls are doing a good job of pushing laggard content providers down this path.[4]

[1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15...

[2] https://github.com/gentoo/gentoo/commit/1265a159743d7f07185a...

[3] https://lists.fedoraproject.org/archives/list/devel@lists.fe...

[4] https://news.ycombinator.com/item?id=38249527

Post reply on HN