Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

github.com

Post reply on HN
Print view View original

11–15 of 15 posts

Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

#11

Post by scott_s »

antixk wrote: post #6
scott_s wrote: post #2

For disclosure, I've worked on TorchCodec. I'm happy to answer any questions!

Hi, In the past I have used NVVideoCodec and VPI for gpu accelerated decoding and processing. What would be torchcodec's appeal here? VPI already provides zero-copy interface with pytorch. Thanks!

1. A higher-level API that better integrates into the PyTorch ecosystem.

2. Ease of going back-and-forth between CPU and GPU; in our experience, there's still a lot of scenarios where CPU decoding makes sense.

3. Audio decoding support.

Please take a look at our tutorials to get a feel for what TorchCodec can do: https://meta-pytorch.org/torchcodec/stable/generated_example...

Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

#12

Post by NicolasHug »

weitendorf wrote: post #8
scott_s wrote: post #2

For disclosure, I've worked on TorchCodec. I'm happy to answer any questions!

> TorchCodec now has a dedicated WavDecoder for decoding WAV files. It bypasses FFmpeg entirely and reads WAV data directly, resulting in significantly faster decoding. I'm working in this area recently and very keen to use this given the claimed performance benefits, but I tried all your links and didn't see any actual performance numbers. Do you have any to share? IMO a fair performance benchmark for those not tied…

We tend to be conservative with the benchmarks results that we make public, because all benchmarks are wrong and unfair - they depend too much on the machine capabilities, on software versions, and on the actual decoding patterns that are relevant for the user - none of which can be controlled or fairly captured in a benchmark. That being said, we've got some benchmarks here, with a script that users can run on their own: https://github.com/meta-pytorch/torchcodec/pull/1474.

Note that TorchCodec relies on FFmpeg libraries, not the FFmpeg binary itself. The new WavDecoder is faster because it bypasses the FFmpeg libraries code, not because it bypasses loading the FFmpeg binary in memory.

Regarding static linking: we stick to dynamic linking to honor the L-GPL license of the FFmpeg libraries. TorchCodec is BSD-licensed, and statically linking against the L-GPL FFmpeg libs would not be compliant. Some libraries dynamically link against FFmpeg while still bundling the FFmpeg libraries as .so files in the Python wheel - whether that's still compliant is honestly unclear to me, so we prefer leaving it up for the user to supply their own FFmpeg via pure dynamic linking.

Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

#13

Post by scott_s »

scott_s wrote: post #10
alphatozeta wrote: post #5

its really fast and the performance is great, but its really unfortunate it requires torch>=2.11 Too many NVIDIA libraries are still using 2.10 or an alpha version of 2.11 that doesn't have c++ methods used by torchcodec's underlying C++ code like use_blob and a few others. I had to fall back to ffmpeg-python unfortunately

You can get older version of TorchCodec that work with older version of PyTorch, but it unfortunately will not have the new features (HDR video decoding; fast Wav decoding) in the latest release. See the compatbility matrix: https://github.com/meta-pytorch/torchcodec#compatibility-wit... Up until recently, TorchCodec releases worked with one-and-only-one version of PyTorch. This is because up until recently, PyTorch…

I was copying URLs fast-and-furious. The PyTorch Stable ABI reference is: https://docs.pytorch.org/docs/2.12/notes/libtorch_stable_abi...

Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

#14

Post by hmaarrfk »

scott_s wrote: post #9
hmaarrfk wrote: post #4

What version of ffmpeg does this use? Last I tried torch tools used really outdated version of ffmpeg at the time of their release.

The one you have installed. :) We don't distribute FFmpeg and instead find your installed version at runtime. We support versions 4 through 8.

That’s a large support matrix. Congratulations.

Do you link to the shared object files? Or do you use subprocess?

Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder

#15

Post by scott_s »

hmaarrfk wrote: post #14
scott_s wrote: post #9

Earlier quoted context omitted.

The one you have installed. :) We don't distribute FFmpeg and instead find your installed version at runtime. We support versions 4 through 8.

That’s a large support matrix. Congratulations. Do you link to the shared object files? Or do you use subprocess?

We dynamically load the shared object files. See: https://github.com/meta-pytorch/torchcodec/blob/8bbce656797c...

Solutions that shell out to the `ffmpeg` binary are not going to perform well.

Post reply on HN