TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
1–10 of 15 posts
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#2For disclosure, I've worked on TorchCodec. I'm happy to answer any questions!
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#3the WAV file decoding perf improvement is also very welcome!
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#4What version of ffmpeg does this use? Last I tried torch tools used really outdated version of ffmpeg at the time of their release.
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#5its really fast and the performance is great, but its really unfortunate it requires torch>=2.11
Too many NVIDIA libraries are still using 2.10 or an alpha version of 2.11 that doesn't have c++ methods used by torchcodec's underlying C++ code like use_blob and a few others. I had to fall back to ffmpeg-python unfortunately
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#6For disclosure, I've worked on TorchCodec. I'm happy to answer any questions!
Hi,
In the past I have used NVVideoCodec and VPI for gpu accelerated decoding and processing. What would be torchcodec's appeal here? VPI already provides zero-copy interface with pytorch.
Thanks!
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#7[deleted]
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#8For disclosure, I've worked on TorchCodec. I'm happy to answer any questions!
> TorchCodec now has a dedicated WavDecoder for decoding WAV files. It bypasses FFmpeg entirely and reads WAV data directly, resulting in significantly faster decoding.
I'm working in this area recently and very keen to use this given the claimed performance benefits, but I tried all your links and didn't see any actual performance numbers. Do you have any to share?
IMO a fair performance benchmark for those not tied to the full pytorch stack would have ffmpeg and the wav already loaded into memory before execution. Given that torchcodec relies on the user-supplied ffmpeg installation I suspect that may not be the case for ffmpeg already, at least not by default.
I understand why meta wouldn't want to do this (then you are inevitably distributing exploitable security vulnerabilities in pytorch, because ffmpeg will probably always have them) but I've been statically linking fmpeg and keeping the binary in-memory while still using separate processes for different batches of audio, with I/O through UDS between the parent and ffmpeg; then the parent does VAD on the pcm on CPU before any further inference. My implementation for static linking is similar to the pattern in https://github.com/amenzhinsky/go-memexec#static-binary - would be interesting to see if this is possible in the pytorch/python ecosystem, or maybe it's already been done.
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#9What version of ffmpeg does this use? Last I tried torch tools used really outdated version of ffmpeg at the time of their release.
The one you have installed. :) We don't distribute FFmpeg and instead find your installed version at runtime. We support versions 4 through 8.
Re: TorchCodec 0.14: HDR Video Decoding for CPU and CUDA, and Fast Wav Decoder
#10its really fast and the performance is great, but its really unfortunate it requires torch>=2.11 Too many NVIDIA libraries are still using 2.10 or an alpha version of 2.11 that doesn't have c++ methods used by torchcodec's underlying C++ code like use_blob and a few others. I had to fall back to ffmpeg-python unfortunately
You can get older version of TorchCodec that work with older version of PyTorch, but it unfortunately will not have the new features (HDR video decoding; fast Wav decoding) in the latest release. See the compatbility matrix: https://github.com/meta-pytorch/torchcodec#compatibility-wit...
Up until recently, TorchCodec releases worked with one-and-only-one version of PyTorch. This is because up until recently, PyTorch did not have a stable ABI, and we needed to pin TorchCodec releases to PyTorch releases. But! PyTorch now has an excellent Stable ABI (https://github.com/meta-pytorch/torchcodec#compatibility-wit..., https://www.youtube.com/watch?v=HNdEmnvMvGE&t=1s) and TorchCodec is taking advantage of that since version 0.12.