Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulkan video extensions for accelerated H.264 and H.265 encode

khronos.org

Post reply on HN
Print view View original

11–20 of 110 posts

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#11

Post by brigade »

jorvi wrote: post #6
shmerl wrote: post #4

Earlier quoted context omitted.

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

Vulkan Video Extensions use the same physical hardware as VAAPI, but with a little bit less of forcing you to do codec-specific nonsense. It should be better in that it's much easier to get a zero-copy Vulkan texture from the decoded frame.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#12

Post by kllrnohj »

shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

> Now we need OBS and Firefox supporting Vulkan video too.

Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging very far behind native APIs that they're already using.

It's not a bad extension necessarily, seems like it'd be a great option for games that want to integrate some cutscenes. But if you have an existing solution, it also doesn't seem valuable to migrate to this, either.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#13

Post by stefan_ »

kllrnohj wrote: post #9
jorvi wrote: post #6

Earlier quoted context omitted.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extens…

It's unhelpful to bundle this into the "GPU" at large and is only true for desktop systems anyway. On mobile, video decoding, GPU and display pipeline are typically different hardware blocks often made by different companies.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#14

Post by brigade »

kllrnohj wrote: post #9
jorvi wrote: post #6

Earlier quoted context omitted.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extens…

> MoltenVK doesn't implement this extension (and probably won't)

Since Khronos appears to have decided to require manual DPB management, this cannot be implemented in HW on macOS by MoltenVK.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#15

Post by theHamsta »

jorvi wrote: post #6
shmerl wrote: post #4

Earlier quoted context omitted.

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

The GPU has also the fixed function video encode like integrated CPUs. Both VAAPI and Vulkan Video address the same hardware. Differences are in the API itself (with Vulkan being more explicit) and potentially in which vendors (or OSS driver developers) support which API in what degree.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#16

Post by deskamess »

jorvi wrote: post #6
shmerl wrote: post #4

Earlier quoted context omitted.

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

Could anyone elaborate on the difference between software decoding and CPU decoding? Software decoding uses the CPU right?

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#17

Post by jdiff »

kllrnohj wrote: post #12
shmerl wrote: post #2

> In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. That's good. By the way, mpv already supports Vulkan video decoding: https://github.com/mpv-player/mpv/issues/11739 Now we need OBS and Firefox supporting Vulkan video too.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#19

Post by mattst88 »

jdiff wrote: post #17
kllrnohj wrote: post #12

Earlier quoted context omitted.

> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…

Firefox still has no support for hardware video decode on Linux. There's all sorts of shims for it, but I've never managed to get them functioning. Maybe Vulkan video decode could be the one agnostic API to rule them all this time, especially if it's more cross-platform than the current situation.

> Firefox still has no support for hardware video decode on Linux.

That's not true at all. See:

https://www.omgubuntu.co.uk/2023/07/firefox-115-intel-gpu-vi... https://www.phoronix.com/news/Mozilla-Firefox-115 https://www.omglinux.com/firefox-hardware-acceleration-raspb...

Re: Vulkan video extensions for accelerated H.264 and H.265 encode

#20

Post by mattst88 »

deskamess wrote: post #16
jorvi wrote: post #6

Earlier quoted context omitted.

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

Could anyone elaborate on the difference between software decoding and CPU decoding? Software decoding uses the CPU right?

That's right. "Software decoding" means the decoding algorithms run on the CPU. (As opposed to "hardware decoding" which typically means the work is done by some fixed-function video units on e.g. a GPU)
Post reply on HN