Live data from Hacker News

FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

phoronix.com

151–160 of 186 posts

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#151

Earlier quoted context omitted.

It seems ffmpeg uses the mailing list patch way of doing "PRs", which is... well it is what it is. It doesn't help me understand the process unless I just go through all the mailing list archives, I guess.

Ugh why? That is so old school

I mean they might be used to doing that as ffmpeg is predating github. (and git.)

as long as it works for them...

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#152
post #128

Earlier quoted context omitted.

> exhaustively finding the best possible combination of QP/resolution pairs for an entire encoding ladder that also optimizes subjective quality – and not just MSE. This is unnecessary if the encoder is well-written. It's like how some people used to run multipass encoders 3 or 4 times just in case the result got better. You only need one analysis pass to find the optimal quality at a bitrate.

Sure, the whole point of CRF is to set a quality target and forget about it, or, with ABR, to be as good as you can with an average bitrate target (under constraints). But you can't do that across resolutions, e.g. do you pick the higher bitrate 360p version, or the lower bitrate 480p one, considering both coding artifacts and upscaling degradation?

At those two resolutions you'd pick the higher resolution one. I agree that generation of codec doesn't scale all the way up to 4K and at that point you might need to make some smart decisions.

I think it should be possible to decide in one shot in the codec though. My memory is that codecs (image and video) have tried implementing scalable resolutions before, but it didn't catch on simply because dropping resolution is almost never better than dropping bitrate.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#153

Earlier quoted context omitted.

> Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v ... and https://netflixtechblog.com/optimized-shot-based-encodes-now ... This is overrated - of course that's how you do it, what else would you do? > Mean-squared-error (MSE), typically used for encoder decisions, is a numbe…

You're letting the video codec make all the decisions for bitrate allocation. Netflix tries to optimize the encoding parameters per shot/scene. from the dynamic optimization article: - A long video sequence is split in shots ("Shots are portions of video with a relatively short duration, coming from the same camera under fairly constant lighting and environment conditions.") - Each shot is encoded multiple times with…

> You're letting the video codec make all the decisions for bitrate allocation. > Netflix tries to optimize the encoding parameters per shot/scene.

That's the problem - if the encoding parameters need to be varied per scene, it means you've defined the wrong parameters. Using a fixed H264 QP is not on the rate-distortion frontier, so don't encode at constant QP then. That's why x264 has a different fixed quality setting called "ratefactor".

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#154

Earlier quoted context omitted.

You're letting the video codec make all the decisions for bitrate allocation. Netflix tries to optimize the encoding parameters per shot/scene. from the dynamic optimization article: - A long video sequence is split in shots ("Shots are portions of video with a relatively short duration, coming from the same camera under fairly constant lighting and environment conditions.") - Each shot is encoded multiple times with…

> You're letting the video codec make all the decisions for bitrate allocation. > Netflix tries to optimize the encoding parameters per shot/scene. That's the problem - if the encoding parameters need to be varied per scene, it means you've defined the wrong parameters. Using a fixed H264 QP is not on the rate-distortion frontier, so don't encode at constant QP then. That's why x264 has a different fixed quality sett…

What about VP9? And any of the other codecs that Netflix uses (I'll assume AV1 is one they currently use)?

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#156
post #56

Earlier quoted context omitted.

your idea also doesn't work with live streaming, and may also not work with inter-frame filters (depending on implementation). nonetheless, this exists already with those limitations: av1an and I believe vapoursynth work more or less the way you describe, except you don't actually need to load every chunk into memory, only the current frames. as I understand, this isn't a major priority for mainstream encoding pipeli…

It can work with live streaming, you just need to add N keyframes of latency. With low-latency livestreaming keyframes are often close together anyways so adding say 4s of latency to get 4x encoding speed may be a good tradeoff.

You can pry vp8 out of my cold dead heands. I'm sorry, but if it takes more than 200ms including network latency it is too slow and video encoding is extremely CPU intensive so exploding your cloud bill is easy.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#158
post #82

Earlier quoted context omitted.

https://www.youtube.com/watch?v=9kaIXkImCAM

"Do you know ffmpeg supports OCR? I haven't found the command yet, but it does support it." This is probably 80% of my experience with ffmpeg, to be honest, but the other 20% is invaluable enough anyway.

I thought this was a joke, but it really exists! https://ffmpeg.org/ffmpeg-filters.html#ocr

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#159

Earlier quoted context omitted.

> Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v ... and https://netflixtechblog.com/optimized-shot-based-encodes-now ... This is overrated - of course that's how you do it, what else would you do? > Mean-squared-error (MSE), typically used for encoder decisions, is a numbe…

You're letting the video codec make all the decisions for bitrate allocation. Netflix tries to optimize the encoding parameters per shot/scene. from the dynamic optimization article: - A long video sequence is split in shots ("Shots are portions of video with a relatively short duration, coming from the same camera under fairly constant lighting and environment conditions.") - Each shot is encoded multiple times with…

Isn't two pass encoding similar? In the first pass you collect statistics you use in the second pass for bandwidth allocation?

Possibly Netflix statistics are way better.

Re: FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

#160
post #132

A theory about this that may also affect other older solid software: the assumptions made on where to optimally "split" a problem for multi-threading/processing has likely changed over time. It wasn't that long ago that reading, processing, and rendering the contents of a single image took a noticeable amount of time. But both hardware and software techniques have gotten significantly faster. What may have made sense…

I assume 1 process with 2 threads takes up less space on the die than 2 processes, both single threaded. If this is true, a threaded solution will always have the edge on performance, even as everything scales ad infinitum.
Post reply on HN