> vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR how to exceed the character limit on your super wide screen with just one function name
[deleted]
Vulkan video extensions for accelerated H.264 and H.265 encode
41–50 of 110 posts
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#42Earlier quoted context omitted.
> Now we need OBS and Firefox supporting Vulkan video too. Why? They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos. Why use Vulkan video extensions? It seems like it just complicates things since MacOS/iOS and Android are unlikely to get these and Vulkan on Windows' is a second-class citizen as well. Not to mention the feature & API availability is lagging v…
> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not…
> One vendor specific API, not "various OS' native APIs".
Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well.
It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video extensions will be the savior on Linux. Or maybe it won't because it won't be anyone's focus of investment since it's largely a Linux-only problem in the first place.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#43> vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR how to exceed the character limit on your super wide screen with just one function name
Eskil Steenberg suggests width of code as a measure of quality. Vulkan wins.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#44Earlier quoted context omitted.
> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not…
> One vendor specific API, not "various OS' native APIs". Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well. It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video…
I'm always annoyed how any Linux media player or encoder needs to bring its own entire media operating system, down to each individual nut and bolt.
On Windows there's Windows Media Foundation and DirectShow that centrally manage everything and also support the "individual nut and bolt" approach. Android has its own central thing (MediaCodec?) that must be used. MacOS and iOS presumably have their own central manager (Quicktime?) too.
But Linux? It doesn't serve as an operating system for media. It's tremendously inconvenient as an admin/user rather than an evangelist.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#45Earlier quoted context omitted.
> One vendor specific API, not "various OS' native APIs". Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well. It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video…
I'm always annoyed how any Linux media player or encoder needs to bring its own entire media operating system, down to each individual nut and bolt. On Windows there's Windows Media Foundation and DirectShow that centrally manage everything and also support the "individual nut and bolt" approach. Android has its own central thing (MediaCodec?) that must be used. MacOS and iOS presumably have their own central manager…
You don't need to implement every nut and bolt in the application. Lot's of useful things can do the heavy lifting (Pipewire, ffmpeg, libplacebo, Mesa and so on). Linux isn't after calling it all using some uniform "DirectFoo" naming scheme, but tools are there.
Comparison is also invalid. Linux as a whole (not the kernel but OS experience) isn't controlled by some Big Brother who decides what and how it's done single mindedly. So such kind of composite result is somewhat expected.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#46Earlier quoted context omitted.
Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.
What of it? So does nvidia-vaapi-driver. They're third party projects, that's very different from "supported by the vendor", and doesn't change the fact that NVIDIA as a vendor offers no support for VA-API. By the way, nouveau's support is currently limited and not useful: https://nouveau.freedesktop.org/VideoAcceleration.html see Video engine support status table, only old GPUs and no H.265 or AV1 support.
> What of it?
It was an answer to this question specifically
> Which 3 vendors are you referring to? Intel, AMD, and who?
I either missed some of the other text in your post or it was added after I started to reply.
> Both official open source drivers (open-gpu-kernel-module)
This is not remotely close to being a complete graphics driver. Most of a GPU driver on Linux is in userspace and there is no official open source user space component.
> Why are you bringing up open source drivers, and what is not practical?
nouveau has never been practical for serious use due to poor performance and mediocre hardware support (as you noted). open-gpu-kernel-module is only practical when paired with a proprietary userspace driver.
Anyway, my original point in all this is that describing VA-API as an Intel vendor specific API is unfair given it has been well supported on AMD GPUs for a long time now and on nouveau it's supported as well as VDPAU (i.e. not very well as you note). I did not intend to imply that it was universal. I didn't even intend to imply that VDPAU is a vendor specific API (though as a decode-only API it's not really a complete replacement).
Intel tried to make va-api the standard for hardware encode and decode on Linux, Nvidia tried to make VDPAU the standard for hardware decode on Linux. Neither was entirely successful. By contrast, NVENC/NVDEC, AMF and the Intel Media SDK (and whatever they replaced this with) never had such ambitions.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#47Earlier quoted context omitted.
On Windows/Mac Firefox uses the system hardware video decoding APIs - DXVA and Video Toolbox.
On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.
But why would Firefox go out of its way to use Vulkan on Windows or macOS when it doesn’t have to because it can just use the native OS functionality and let the OS worry about implementation details?
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#48Earlier quoted context omitted.
> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not…
> One vendor specific API, not "various OS' native APIs". Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well. It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video…
What is "incorrect"? The full sentence that you conveniently chose to cut in the middle before quoting (apparently to fit into some pessimistic forecast about the significance of Linux desktop) reads
> Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA.
(emphasis added)
You further wrote:
> Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor.
And? None of those APIs are cross-platform. Vulkan Video will eventually allow developers (including Firefox developers) to write a single code path for video to cover a wide range of platforms and vendors (likely with the exception of walled gardens like Apple-land, although someone might find a way to support like via a wrapper like MoltenVk for Vulkan).
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#49Earlier quoted context omitted.
On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.
But why would Firefox go out of its way to use Vulkan on Windows or macOS when it doesn’t have to because it can just use the native OS functionality and let the OS worry about implementation details?
I don't really care much about Windows and macOS. But I'd like to see it using Vulkan video on Linux in addition or instead of VAAPI.
As pointed out elsewhere, it's not possible on macOS either way. Apple are too stuck up not supporting Vulkan and it won't work with MoltenVK.
Re: Vulkan video extensions for accelerated H.264 and H.265 encode
#50Earlier quoted context omitted.
Is there a reason we don't have parallelizable video formats?
Modern video codecs use motion prediction algorithms to encode the information between key frames which depends on the entire frame and the ones immediately preceding it. You can encode the independent groups of pictures (the key frame and all the subsequent predictive frames until the next key frame) in parallel but at least with 4k video you hit memory bandwidth limitations quite quickly.
Also, to get best compression you don't want to do key frames at even intervals, but rather when it pays off (ie large changes between two frames, like a scene change).