Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

phoronix.com

Post reply on HN
Print view View original

101–110 of 186 posts

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#101

Post by airstrike »

motoboi wrote: post #45
bigbillheck wrote: post #12

Earlier quoted context omitted.

Why on earth would you possibly think that?

I'm quite confused by the answers I got from this thread. Haven't you tried refactoring with gpt-4 yet?

Some people get significance from their ability to write code. To them, admitting an LLM can (or will soon be able to) do their work inflicts cognitive dissonance, so they refuse to believe it. Some refuse to even try it—not realizing that refusing to engage does nothing to hinder the advancement of the tool they fear.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#102

Post by astrange »

cudder wrote: post #58
kevincox wrote: post #46

I've always wondered if better multi-core performance can come from processing different keyframe segments separately. IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality/efficiency cost and requires…

I know next to nothing about video encoders, and in my naive mind I absolutely thought that parallelism would work just like you suggested it should. It sounds absolutely wild to me that they're splitting single frames into multiple segments. Merging work from different threads for every single frame sounds wasteful somehow. But I guess it works, if that's how everybody does it. TIL!

Most people concerned about encoding performance are doing livestreaming and so they can't accept any additional latency. Splitting a frame into independent segments (called "slices") doesn't add latency / can even reduce it, and it recovers from data corruption a bit better, so that's usually done at the cost of some compression efficiency.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#103

Post by ajhai »

This will hopefully improve the startup times for FFmpeg when streaming from virtual display buffers. We use FFmpeg in LLMStack (low-code framework to build and run LLM agents) to stream browser video. We use playwright to automate browser interactions and provide that as tool to the LLM. When this tool is invoked, we stream the video of these browser interactions with FFmpeg by streaming the virtual display buffer the browser is using.

There is a noticeable delay booting up this pipeline for each tool invoke right now. We are working on putting in some optimizations but improvements in FFmpeg will definitely help. https://github.com/trypromptly/LLMStack is the project repo for the curious.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#104

Post by PatronBernard »

kevincox wrote: post #46

I've always wondered if better multi-core performance can come from processing different keyframe segments separately. IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality/efficiency cost and requires…

IIUC - International Islamic University Chittagong?

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#105

Post by astrange »

mrintegrity wrote: post #21
motoboi wrote: post #3

It’s nuts to think that in the near future LLM will be able to do that refactoring in seconds. All we need is enough context window.

Parent post is getting down voted to oblivion but it seems a reasonable belief for someone who is not highly engaged with AI. I have only the vaguest understanding of how it works (and it's probably wrong) and to my layman mind it also seems like a totally fair assumption, based on experience as a user and the constant flood of news. Please explain why the suggestion that a future AI / sufficiently advanced LLM could…

The ffmpeg tests take a lot more than a few seconds to run, and an AI god is still going to have trouble debugging multithreaded code.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#106

Post by ElijahLynn »

nerpderp82 wrote: post #34
whalesalad wrote: post #32

Earlier quoted context omitted.

I'm dying.

https://www.youtube.com/watch?v=9kaIXkImCAM

That was one of the funniest things I've seen in a while!!!! I had to stop drinking my decaf for fear of spitting it all over my computer I was laughing out loud so much!

(ps: and no, it's not Rick Astley/Never Gonna Give You Up)

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#107

Post by ElijahLynn »

whalesalad wrote: post #41
nerpderp82 wrote: post #34

Earlier quoted context omitted.

https://www.youtube.com/watch?v=9kaIXkImCAM

I'm glad we've reached a point where there is quality parody content online for our industry.

THIS!!! It was so refreshing!

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#108

Post by astrange »

danielrhodes wrote: post #76
seeknotfind wrote: post #59

Earlier quoted context omitted.

Video codecs often encode the delta from the previous frame, and because this delta is often small, it's efficient to do it this way. If each thread needed to process the frame separately, you would need to make significant changes to the codec, and I hypothesize it would cause the video stream to be bigger in size.

Isn't that delta partially based on the last keyframe? I guess it would be codec dependent, but my understanding is that keyframes are like a synchronization mechanism where the decoder catches up to where it should be in time.

In most codecs the entropy coder doesn't reset across frames, so there is enough freedom that you can do multithreaded decoding. ffmpeg has frame-based and slice-based threading for this.

It also has a lossless codec ffv1 where the entropy coder doesn't reset, so it truly can't be multithreaded.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#109

Post by smabie »

bigbillheck wrote: post #92
motoboi wrote: post #45

Earlier quoted context omitted.

I'm quite confused by the answers I got from this thread. Haven't you tried refactoring with gpt-4 yet?

> Haven't you tried refactoring with gpt-4 yet? I most certainly have not. At work, I do greenfield development in a specialized problem domain, and I would not trust a model (or, for that matter, a junior developer) to do any kind of refactor in an acceptable manner. (That aside, there's no way I'm goingto expose company code to any sort of outside party without the approval of upper management). At home, I program…

Oh yea your work is far too sophisticated for a llm, got it

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#110

Post by nolist_policy »

PatronBernard wrote: post #104
kevincox wrote: post #46

I've always wondered if better multi-core performance can come from processing different keyframe segments separately. IIUC all current encoders that support parallelism work by multiple threads working on the same frame at the same time. Often times the frame is split into regions and each thread focuses on a specific region of the frame. This approach can have a (usually small) quality/efficiency cost and requires…

IIUC - International Islamic University Chittagong?

IIUC - If I understand correctly.
Post reply on HN