Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulkan video extensions for accelerated H.264 and H.265 encode

khronos.org

Post reply on HN
Print view View original

21–30 of 110 posts

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#21

Post by cesarb »

mattst88 wrote: post #19
jdiff wrote: post #17

Earlier quoted context omitted.

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

> Firefox still has no support for hardware video decode on Linux. That's not true at all. See: https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

How can I confirm whether Firefox is actually using hardware decode for videos on my machine for a given codec? Is there a magic keyword I should search for within the about:support page, or is it recorded somewhere else? Mine shows for instance "VP9_HW_DECODE default available", does this means it's using hardware decode for VP9, or merely that it might be possible for my Firefox version?

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#22

Post by shmerl »

kllrnohj wrote: post #12
shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

I think a very simple reason is I'd rather rely on Vulkan driver for faster iteration, than on VAAPI. It's historically moving much slower since it first needs Intel to pull weight, then AMD to implement their side for example.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#23

Post by shmerl »

jdiff wrote: post #17
kllrnohj wrote: post #12

Earlier quoted context omitted.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

Firefox has both decoding and encoding already supported via ffmpeg+vaapi. What would be nice to see is ffmpeg+vulkan path for it.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#24

Post by shmerl »

cesarb wrote: post #21
mattst88 wrote: post #19

Earlier quoted context omitted.

> Firefox still has no support for hardware video decode on Linux. That's not true at all. See: https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

How can I confirm whether Firefox is actually using hardware decode for videos on my machine for a given codec? Is there a magic keyword I should search for within the about:support page, or is it recorded somewhere else? Mine shows for instance "VP9_HW_DECODE default available", does this means it's using hardware decode for VP9, or merely that it might be possible for my Firefox version?

At least with AMD GPU, an easy way to do it is to check usage of VCN.

Something like this (assuming your GPU index is 0):

    watch -n 1 sudo cat /sys/kernel/debug/dri/0/amdgpu_pm_info
You should see there:

    VCN: Enabled
If GPU accelerated video is being played.

I think you might need to set this flag to true in Firefox's about:config (or it might be not needed anymore):

    media.ffmpeg.vaapi.enabled

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#25

Post by shmerl »

jorvi wrote: post #6
shmerl wrote: post #4

Earlier quoted context omitted.

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

I think not being tied to VAAPI reduces the chain of development (like you don't need to wait for Intel to add something first since they are the primary developers of VAAPI in general).

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#26

Post by shmerl »

dishsoap wrote: post #18

So to clarify, this still leverages the fixed function hardware video encoders present alongside many GPUs from eg. Intel/NVIDIA/AMD, and would not (for example) be possible to use on just a GPU itself (without said encoding hardware)?

It is for using GPU video related ASICs (same as VAAPI let's say). If you want to use shaders for some video filters, scaling and such - you don't need those extensions.

mpv for example can use Vulkan for both at the same time.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#27

Post by jorvi »

kllrnohj wrote: post #9
jorvi wrote: post #6

Earlier quoted context omitted.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extens…

Huh, I stand corrected. I always figured they stuffed it in its own little bit of silicon. Thanks for clearing it up!

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#28

Post by tagrun »

kllrnohj wrote: post #12
shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos

One vendor specific API, not "various OS' native APIs".

Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not yet reliable as the officially supported VDPAU or NVDEC, and is not included in official linux package repositories.)

Intel has VA-API, AMD has AMF, and NVIDIA has VDPAU which is being replaced by NVDEC/NVENC.

The idea behind Vulkan Video Extensions is to have a vendor independent and cross-platform video API.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#29

Post by mikepavone »

tagrun wrote: post #28
kllrnohj wrote: post #12

Earlier quoted context omitted.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not…

> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA.

Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#30

Post by tagrun »

mikepavone wrote: post #29
tagrun wrote: post #28

Earlier quoted context omitted.

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not…

> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA. Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers onl…

You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then?

Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like the list of available codecs and encoding features).

You are free to call these vendor specific APIs for what they are or something else, but the reality has been that there is no single video API officially supported by Intel, AMD and NVIDIA. This changed with Vulkan Video.

But Vulkan Video isn't just about desktop: mobile devices, Raspberry Pi, etc. are expected to get on board with it eventually, just like they did with Vulkan.

> It's supported by the open source drivers for GPUs from all 3 vendors.

Which 3 vendors are you referring to? Intel, AMD, and who?

> It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc

Why are you bringing up open source drivers, and what is not practical? Both official open source drivers (open-gpu-kernel-module) and unofficial open source drivers (nouveau, through binary firmware) support VDPAU. However, NVIDIA's drivers (open source or binary) does not support VA-API.

Post reply on HN