This isn't strictly Netflix per se, it's part of the AV1 codec itself, e.g. https://github.com/BlueSwordM/SVT-AV1/blob/master/Docs/Appen...
Yes, I was not strictly correct, it is a feature of AV1, but Netflix played an active role in its development, in rolling out the first implementation, and in AV1 codec development overall.
Prior codecs all had film grain synthesis too (at least back to H264), but nobody used it. Partly because it was obviously artificial and partly because it was too expensive to render, since you had to copy each frame to apply effects to it.
Content delivery costs a lot for streaming services. After content is produced, this is basically the only remaining cost. It’s not surprising that they would go to extreme measures in reducing bitrate. That’s why, presumably, Netflix came up with the algorithm for removing camera grain and adding synthetically generated noise on the client[0], and why YouTube shorts were recently in the news for using extreme denois…
It feels to me like there are two different things going on: 1. Video codecs like the denoise, compress, synthetic grain approach because their purpose is to get the perceptually-closest video to the original with a given number of bits. I think we should be happy to spend the bits on more perceptually useful information. Certainly I am happy with this. 2. Streaming services want to send as few bytes as they can get…
For #1 the problem with keeping grain in the compressed video is that it doesn't follow the motion of the scene so it makes it much more expensive to code future frames.
Will streaming services ever stop over-compressing their content? I have a top-of-the-line 4K TV and gigabit internet, yet the compression artifacts make everything look like putty. Honestly, the best picture quality I’ve ever seen was over 20 years ago using simple digital rabbit ears. You especially notice the compression on gradients and in dark movie scenes. And yes — my TV is fully calibrated, and I’m paying for…
> You especially notice the compression on gradients and in dark movie scenes. That's not a correctly calibrated TV. The contrast is tuned WAY up. People do that to see what's going on in the dark, but you aren't meant to really be able to see those colors. That's why it's a big dark blob. It's supposed to be barely visible on a well calibrated display. A lot of video codecs will erase details in dark scenes because…
Video codecs aren't tuned for any particular TV calibration. They probably should be because it is easier to spot single bit differences in dark scenes, because the relative error is so high.
The issue is just that we don't code video with nearly enough bits. It's actually less than 8-bit since it only uses 16-235.
> 1. camera manufacturers and film crews both do their best to produce a noise-free image 2. in post-production, they add fake noise to the image so it looks more "cinematic" This is patently wrong. The rest builds up on this false premise.
1.1: Google some low-light performance reviews of cinema cameras - you'll see that ISO noise is decreasing with every generation and that some cameras (like from Sony) have that as a selling feature. 1.2.: Google "how to shoot a night scene" or something like that. You'll find most advice goes something along the lines of "don't crank the ISO up, add artificial lighting to brighten shadows instead". When given a choi…
I am reasonably confident that I know this subject area better than one could learn from three Google searches. “ISO noise” is not even a real term; you are talking about digital sensor noise, which can further be luminance noise or colour noise. Opinions about luma noise being “unaesthetic” are highly subjective, depending a lot on stylistic choices and specific sensor, and some would say luma noise on professional sensors has not been ugly for more than a decade. Generally, the main reason I don’t think your comments should be taken seriously is because you are basically talking only about a subset of photography, digital photography, while failing to acknowledge that (works are still shot on film in 2025 and will be in years to come).
It'd be really cool if we had 'upgradable codec FPGAs' in our machines that you could just use flash to the newest codec... but that'd probably be noticeably more expensive, and also not really in the interest of the manufacturers, who want to have reasons to sell new chips.
> FPGAs' in our machines that you could just use flash to the newest codec They're called GPUs... They're ASICs rather than FPGAs, but it's easy to update the driver software to handle new video codecs. The difficulty is motivating GPU manufacturers to do so... They'd rather sell you a new one with newer codec support as a feature.
GPU video decoding doesn't use the graphics cores. It just happens to be on the GPU and is not upgradeable.
Will streaming services ever stop over-compressing their content? I have a top-of-the-line 4K TV and gigabit internet, yet the compression artifacts make everything look like putty. Honestly, the best picture quality I’ve ever seen was over 20 years ago using simple digital rabbit ears. You especially notice the compression on gradients and in dark movie scenes. And yes — my TV is fully calibrated, and I’m paying for…
> You especially notice the compression on gradients and in dark movie scenes. That can happen at even the highest bitrates if "HDR" is not enabled in the video codec. Related video: https://www.youtube.com/watch?v=h9j89L8eQQk
Whoa. That is the best thing i watched on YouTube in a long time. Thank you.
Yet I still only got hardware support for it on my first devices last year. The downside of "rapid" iteration on video codecs is that content needs to always be stored in multiple formats (or alternatively battery life on the client suffers from software playback, which is the route e.g. Youtube seems to be preferring).
Unless they create a codec that GPUs are naturally good at, we will inevitably always be a couple of hardware cycles behind.
You can't do this because GPUs are parallel and decompression cannot be parallel. If there's any parallelism it means it's not compressed as much as it could be.
Content delivery costs a lot for streaming services. After content is produced, this is basically the only remaining cost. It’s not surprising that they would go to extreme measures in reducing bitrate. That’s why, presumably, Netflix came up with the algorithm for removing camera grain and adding synthetically generated noise on the client[0], and why YouTube shorts were recently in the news for using extreme denois…
It feels to me like there are two different things going on: 1. Video codecs like the denoise, compress, synthetic grain approach because their purpose is to get the perceptually-closest video to the original with a given number of bits. I think we should be happy to spend the bits on more perceptually useful information. Certainly I am happy with this. 2. Streaming services want to send as few bytes as they can get…
I disagree, because 1) complete denoising is simply impossible while preserving fine detail and 2) noise is a serious artistic choice—just like anamorphic flare, lens FOV with any distortion artifacts, chromatic aberration, etc. Even if it is synthetic film grain that is added in post, that has been somebody’s artful decision; removing it and simulating noise on the client butchers the work.
Content delivery costs a lot for streaming services. After content is produced, this is basically the only remaining cost. It’s not surprising that they would go to extreme measures in reducing bitrate. That’s why, presumably, Netflix came up with the algorithm for removing camera grain and adding synthetically generated noise on the client[0], and why YouTube shorts were recently in the news for using extreme denois…
I recall Netflix saying that streaming cost was nothing compared to all other costs.
I am curious to see their breakdown. It seems very counter-intuitive of them to invest so much into reducing bitrate if the cost of delivery is negligible. R&D and codec design efforts cost money and running more optimized codecs and aggressive denoising cost compute.