Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

phoronix.com

Post reply on HN
Print view View original

71–80 of 186 posts

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#71

Post by kevincox »

Hello71 wrote: post #56
kevincox wrote: post #46

I've always wondered if better multi-core performance can come from processing different keyframe segments separately. IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality/efficiency cost and requires…

your idea also doesn't work with live streaming, and may also not work with inter-frame filters (depending on implementation). nonetheless, this exists already with those limitations: av1an and I believe vapoursynth work more or less the way you describe, except you don't actually need to load every chunk into memory, only the current frames. as I understand, this isn't a major priority for mainstream encoding pipeli…

> except you don't actually need to load every chunk into memory, only the current frames.

That's a good point. In the general case of reading from a pipe you need to buffer it somewhere. But for file-based inputs the buffering concerns aren't relevant, just the working memory.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#72

Post by _8j50 »

hereme888 wrote: post #31
_8j50 wrote: post #20

When I stream 4k from my laptop ffmpeg gets very intense about cpu usage to the point fans are constantly at high speed and it's distracting. I hope this helps in some way. I have a fairly decent specces mid-tier laptop.

I believe ffmpeg can be compiled to support the GPU, if your laptop has one. It works at least for CUDA-enabled GPUs ( https://docs.nvidia.com/video-technologies/video-codec-sdk/1... ) Talk with ChatGPT about it and see if you can do it.

Thanks, I don't think it has a gpu separate from that is cuda-enabled but I have other systems that do, will look into it.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#73

Post by ctoth »

kcbanner wrote: post #55
ctoth wrote: post #51

Earlier quoted context omitted.

Ah, we're having some classic goalpost moving! > Because refactoring requires understanding, which LLMs completely lack. > Cleaning up code also follows some well established patterns, performance work is much less pattern-y. Just as writing shitty react apps follow patterns, low-level performance and concurrency work also follow patterns. See [0] for a sample. > I bet you need 10 or 100 times more understanding Okay…

> Because refactoring requires understanding, which LLMs completely lack. It's obvious from context here that the refactoring that was mentioned was specifically around concurrency, not simply cleaning up code.

So if I show you an LLM implementing concurrency, will you concede the point? Is this your true objection?

https://chat.openai.com/share/7c41f59a-c21c-4abd-876c-c95647...

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#74

Post by satvikpendem »

atrus wrote: post #60
not_the_fda wrote: post #33

Earlier quoted context omitted.

Because refactoring requires understanding, which LLMs completely lack.

Chess requires understanding, which computers lack. Go requires understanding, which computers lack. X requires Y which AI technology today lacks. AI is a constantly moving goalpost it seems.

> AI is a constantly moving goalpost it seems.

alwayshasbeen.png

> The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not "real" intelligence.[1] > Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'."[2] Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'"[3]

> "AI is whatever hasn't been done yet."

> —Larry Tesler

https://en.wikipedia.org/wiki/AI_effect

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#75

Post by naikrovek »

tetris11 wrote: post #66
naikrovek wrote: post #57

Earlier quoted context omitted.

I don't understand why you would want to piggyback on this story to say this. are people just itching for reasons to dive into show & tell or to wax poetic about how they have solved the problem for years ? I really don't understand people at all, because I don't understand why people do this. and I'm sure I've done it, too.

There is hype for FEAT. People who have achieved similar FEAT perk up their heads but say nothing. Hype for FEAT is beyond sensibility. People with similar FEAT are bristled by this and wish that their projects received even a fraction of FEAT's hype. I think it's normal.

not gonna define FEAT, then? ok.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#76

Post by danielrhodes »

seeknotfind wrote: post #59
kevincox wrote: post #46

I've always wondered if better multi-core performance can come from processing different keyframe segments separately. IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality/efficiency cost and requires…

Video codecs often encode the delta from the previous frame, and because this delta is often small, it's efficient to do it this way. If each thread needed to process the frame separately, you would need to make significant changes to the codec, and I hypothesize it would cause the video stream to be bigger in size.

Isn't that delta partially based on the last keyframe? I guess it would be codec dependent, but my understanding is that keyframes are like a synchronization mechanism where the decoder catches up to where it should be in time.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#77

Post by tetris11 »

naikrovek wrote: post #75
tetris11 wrote: post #66

Earlier quoted context omitted.

There is hype for FEAT. People who have achieved similar FEAT perk up their heads but say nothing. Hype for FEAT is beyond sensibility. People with similar FEAT are bristled by this and wish that their projects received even a fraction of FEAT's hype. I think it's normal.

not gonna define FEAT, then? ok.

...in this case, multi-threading. In other cases; AI workflows that others commercialize, a new type system in a language that already exists in another, a new sampling algorithm that has already existed by another name for decades, a permaculture innovation that farmers have been using for aeons, the list goes on...

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#78

Post by canucker2016 »

sanitycheck wrote: post #29
The_Colonel wrote: post #19

Earlier quoted context omitted.

I assume they re-compress for each resolution / format, quite possibly they also have different bitrate levels per resolution. Potentially even variants tweaked for certain classes of device (in cases this is not already covered by combination of format/resolution/bitrate). I would also assume they re-compress with new advances in video processing (things like HDR, improved compression). Also, their devs likely want…

They'll be doing VBR encodes to DASH, HLS & (I guess still) MSS which covers the resolutions & formats... DRM will be what prevents high res content from working on some "less-trusted" platforms so the same encodes should work. (Plus a couple more "legacy" encodes with PIFF instead of CENC for ancient devices, probably.) New tech advances, sure, they probably do re-encode everything sometimes - even knocking a few MB…

Why bring up assumptions/suppositions about Netflix's encoding process?

Their tech blog and tech presentations discuss many of the requirements and steps involved for encoding source media to stream to all the devices that Netflix supports.

The Netflix tech blog: https://netflixtechblog.com/ or https://netflixtechblog.medium.com/

Netflix seems to use AWS CPU+GPU for encoding, whereas YouTube has gone to the expense of producing an ASIC to do much of their encoding.

2015 blog entry about their video encoding pipeline: https://netflixtechblog.com/high-quality-video-encoding-at-s...

2021 presentation of their media encoding pipeline: https://www.infoq.com/presentations/video-encoding-netflix/

An example of their FFmpeg usage - a neural-net video frame downscaler: https://netflixtechblog.com/for-your-eyes-only-improving-net...

Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v... and https://netflixtechblog.com/optimized-shot-based-encodes-now...

Netflix developed an algorithm for determining video quality - VMAF, which helps determine their encoding decisions: https://netflixtechblog.com/toward-a-practical-perceptual-vi..., https://netflixtechblog.com/vmaf-the-journey-continues-44b51..., https://netflixtechblog.com/toward-a-better-quality-metric-f...

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#79

Post by mort96 »

kevincox wrote: post #65
Hello71 wrote: post #56

Earlier quoted context omitted.

your idea also doesn't work with live streaming, and may also not work with inter-frame filters (depending on implementation). nonetheless, this exists already with those limitations: av1an and I believe vapoursynth work more or less the way you describe, except you don't actually need to load every chunk into memory, only the current frames. as I understand, this isn't a major priority for mainstream encoding pipeli…

It can work with live streaming, you just need to add N keyframes of latency. With low-latency livestreaming keyframes are often close together anyways so adding say 4s of latency to get 4x encoding speed may be a good tradeoff.

Well, you don't add 4s of latency for 4x encoding speed though. You add 4s of latency for very marginal quality/efficiency improvement and significant encoder simplification, because the baseline is current frame-parallel encoders, not sequential encoders.

Plus, computers aren't quad cores any more, people with powerful streaming rigs probably have 8 or 16 cores; and key frames aren't every second. Suddenly you're in this hellish world where you have to balance latency, CPU utilization and encoding efficiency. 16 cores at a not-so-great 8 seconds of extra latency means terrible efficiency with a key frame every 0.5 second. 16 cores at good efficiency (say, 4 seconds between key frames) means terrible 64 second of extra latency.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#80

Post by actionfromafar »

seeknotfind wrote: post #68
keehun wrote: post #61

Earlier quoted context omitted.

The parent comment referred to "keyframes" instead of just "frames". Keyframes—unlike normal frames—encode the full image. That is done in case the "delta" you mentioned could be dropped in a stream ending up with strange artifacts in the resulting video output. Keyframes are where the codec gets to press "reset".

Oh right. For non realtime, if you're not IO bound, this is better. Though I'd wonder how portable the codec code itself would be.

The encoder has a lot of freedom in how it arrives at the encoded data.
Post reply on HN