Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

phoronix.com

Post reply on HN
Print view View original

11–20 of 186 posts

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#11

Post by sanitycheck »

bsdpufferfish wrote: post #5

If I'm operating a cloud service like Netflix, then I'm already running thousands of ffmpeg processes on each machine. In other words, it's already a multi-core job.

Curious, what would that many ffmpeg processes be doing at Netflix? I assume new VOD content gets encoded once per format, and the amount of new content added per day is not gigantic.

Agree with the general premise, of course, if I've got 10 different videos encoding at once then I don't need additional efficiency because the CPU's already maxed out.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#13

Post by The_Colonel »

bsdpufferfish wrote: post #5

If I'm operating a cloud service like Netflix, then I'm already running thousands of ffmpeg processes on each machine. In other words, it's already a multi-core job.

I guess it's irrelevant for Netflix then*. But it sounds great for the remaining 99.99%.

* I would be very surprised if Netflix even uses vanilla ffmpeg

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#14

Post by tetris11 »

brucethemoose2 wrote: post #8

Meanwhile, I've been enjoying threaded filter processing in VapourSynth for nearly a decade. Not that this isn't great. Its fantastic. But TBH its not really going to change my workflow of VapourSynth preprocessing + av1an encoding for "quality" video encodes.

I believe I have too with gstreamer's pipe framework for threading, but ffmpeg's syntax has stuck in my mind far longer than any of the elaborate setups I built with gstreamer. I'm excited for this development

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#15

Post by dylan604 »

brucethemoose2 wrote: post #8

Meanwhile, I've been enjoying threaded filter processing in VapourSynth for nearly a decade. Not that this isn't great. Its fantastic. But TBH its not really going to change my workflow of VapourSynth preprocessing + av1an encoding for "quality" video encodes.

FFMPEG does so much more than just video encoding. I use ffmpeg all day every day, and only a fraction of the time do I actually make a video.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#16

Post by bsdpufferfish »

The_Colonel wrote: post #13
bsdpufferfish wrote: post #5

If I'm operating a cloud service like Netflix, then I'm already running thousands of ffmpeg processes on each machine. In other words, it's already a multi-core job.

I guess it's irrelevant for Netflix then*. But it sounds great for the remaining 99.99%. * I would be very surprised if Netflix even uses vanilla ffmpeg

> But it sounds great for the remaining 99.99%.

I believe the vast majority of ffmpeg usages are web services, or one off encodings.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#17

Post by m3kw9 »

dylan604 wrote: post #15
brucethemoose2 wrote: post #8

Meanwhile, I've been enjoying threaded filter processing in VapourSynth for nearly a decade. Not that this isn't great. Its fantastic. But TBH its not really going to change my workflow of VapourSynth preprocessing + av1an encoding for "quality" video encodes.

FFMPEG does so much more than just video encoding. I use ffmpeg all day every day, and only a fraction of the time do I actually make a video.

Like what do you do?

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#18

Post by asveikau »

sanitycheck wrote: post #11
bsdpufferfish wrote: post #5

If I'm operating a cloud service like Netflix, then I'm already running thousands of ffmpeg processes on each machine. In other words, it's already a multi-core job.

Curious, what would that many ffmpeg processes be doing at Netflix? I assume new VOD content gets encoded once per format, and the amount of new content added per day is not gigantic. Agree with the general premise, of course, if I've got 10 different videos encoding at once then I don't need additional efficiency because the CPU's already maxed out.

Probably a lot more than once when you consider that different devices have different capabilities, and that they might stream you different bitrates depending on conditions like your network capability, screen resolution, how much you've paid them..

You could also imagine they might apply some kind of heuristic to decide to re-encode something based on some condition... Like fine tune encoder settings when a title becomes popular. No idea if they do that, just using some imagination.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#19

Post by The_Colonel »

sanitycheck wrote: post #11
bsdpufferfish wrote: post #5

If I'm operating a cloud service like Netflix, then I'm already running thousands of ffmpeg processes on each machine. In other words, it's already a multi-core job.

Curious, what would that many ffmpeg processes be doing at Netflix? I assume new VOD content gets encoded once per format, and the amount of new content added per day is not gigantic. Agree with the general premise, of course, if I've got 10 different videos encoding at once then I don't need additional efficiency because the CPU's already maxed out.

I assume they re-compress for each resolution / format, quite possibly they also have different bitrate levels per resolution. Potentially even variants tweaked for certain classes of device (in cases this is not already covered by combination of format/resolution/bitrate). I would also assume they re-compress with new advances in video processing (things like HDR, improved compression).

Also, their devs likely want fast feedback on changes - I imagine they might have CI running changes on some standard movies, checking various stats (like SNR) for regressions. Everybody loves if their CI finishes fast, so you might want to compress even a single movie in multiple threads.

Post reply on HN