Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AV1@Scale: Film Grain Synthesis, The Awakening

netflixtechblog.com

Post reply on HN
Print view View original

121–130 of 230 posts

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#121

Post by zoky »

kderbe wrote: post #107
crazygringo wrote: post #96

This fails to acknowledge that synthesized noise can lack the detail and information in the original noise . When you watch a high-quality encode that includes the actual noise, there is a startling increase in resolution from seeing a still to seeing the video. The noise is effectively dancing over a signal, and at 24 fps the signal is still perfectly clear behind it. Whereas if you lossily encode a still that disca…

Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine. Regarding aesthetics, I don't think AV1 synthesized grain takes into account the size of the grains in the source video, so chunky grain from an old film source, with its big s…

> Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely)

That might seem like a reasonable assumption, but in practice it’s not really the case. Due to nonlinear response curves, adding noise to a bright part of an image has far less effect than a darker part. If the image is completely blown out the grain may not be discernible at all. So practically speaking, grain does travel with objects in a scene.

This means detail is indeed encoded in grain to an extent. If you algorithmically denoise an image and then subtract the result from the original to get only the grain, you can easily see “ghost” patterns in the grain that reflect the original image. This represents lost image data that cannot be recovered by adding synthetic grain.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#122

Post by creato »

zoky wrote: post #121
kderbe wrote: post #107

Earlier quoted context omitted.

Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine. Regarding aesthetics, I don't think AV1 synthesized grain takes into account the size of the grains in the source video, so chunky grain from an old film source, with its big s…

> Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely) That might seem like a reasonable assumption, but in practice it’s not really the case. Due to nonlinear response curves, adding noise to a bright part of an image has far less effect than a darker part. If the image is completely blown out the grain may not be discernible at all. S…

> If you algorithmically denoise an image and then subtract the result from the original to get only the grain, you can easily see “ghost” patterns in the grain that reflect the original image. This represents lost image data that cannot be recovered by adding synthetic grain.

The synthesized grain is dependent on the brightness. If you were to just replace the frames with the synthesized grain described in the OP post instead of adding it, you would see something very similar.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#123

Post by plorkyeran »

kibwen wrote: post #46
eviks wrote: post #21

> Picture this: you’re watching a classic film, and the subtle dance of film grain adds a layer of authenticity and nostalgia to every scene It just adds visual noise that obscures details of the authentic scene, and nothing prevents nostalgia from being tied to many of the more prominent visual cues like old actors or your own old memories from when you watched it first... > contributing to [film's] realism But ther…

Documentaries might care about accurately representing reality. For every other cinematic genre, "authenticity" is not an inherent goal. If film grain is part of the director's vision, then that's just as valid as a choice to have dramatic non-diagetic music playing in the background of a scene (which is highly inauthentic, but also highly effective at evoking emotion, which is the point of art).

I don't see how that is supposed to be relevant to the article's claim that grain adds authenticity and realism. Someone claiming that non-diegetic music adds to authenticity would also be weird.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#124

Post by crazygringo »

Wowfunhappy wrote: post #101
crazygringo wrote: post #97

Earlier quoted context omitted.

> my real eyes don’t have grain. They definitely do at night when it's dark out. There's a kind of "sparkling" or "static" that comes in faint light. Fortunately, our eyes have way better sensitivity than cameras. But the "realism" just comes from how it was captured using the technology of the day. It's no different from phonograph hiss or the way a CRT signal blurs. The idea is to be "real" to the technology that t…

Your Van Gogh analogy makes sense for old movies. It doesn't quite explain why we're still adding grain to new movies, except for those few which are purposefully evoking older movies.

We don't use obvious grain, usually. That's generally precisely to evoke something about the past -- for flashbacks, a period look, etc. A sense of grittiness like 70s movies, etc.

On the other hand, a small amount of constant grain or noise is intentionally often introduced because otherwise images feel too static and end up looking almost fake. Similarly, dithering is intentionally added to audio, like mastering CD's or tracks. It helps prevent artifacts in video and in audio.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#125

Post by crazygringo »

kderbe wrote: post #107
crazygringo wrote: post #96

This fails to acknowledge that synthesized noise can lack the detail and information in the original noise . When you watch a high-quality encode that includes the actual noise, there is a startling increase in resolution from seeing a still to seeing the video. The noise is effectively dancing over a signal, and at 24 fps the signal is still perfectly clear behind it. Whereas if you lossily encode a still that disca…

Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine. Regarding aesthetics, I don't think AV1 synthesized grain takes into account the size of the grains in the source video, so chunky grain from an old film source, with its big s…

> Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine.

Sorry if I wasn't clear -- I was referring to the underlying objects moving. The codec is trying to capture those details, the same way our eye does.

But regardless of that, you absolutely cannot compare stills. Stills do not allow you to compare against the detail that is only visible over a number of frames.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#126

Post by magicalhippo »

kderbe wrote: post #118
alright2565 wrote: post #113

Earlier quoted context omitted.

I think you've missed the point here: the noise in the originals acts as dithering, and increases the resolution of the original video. This is similar to the noise introduced intentionally in astronomy[1] and in signal processing[2]. Smoothing the noise out doesn't make use of that additional resolution, unless the smoothing happens over the time axis as well. Perfectly replicating the noise doesn't help in this sit…

Your first link doesn't seem to be about introducing noise, but removing it by averaging the value of multiple captures. The second is to mask quantizer-correlated noise in audio, which I'd compare to spatial masking of banding artifacts in video. Noise is reduced to make the frame more compressible. This reduces the resolution of the original only because it inevitably removes some of the signal that can't be differ…

Here's[1] a more concrete source, which summarizes dithering in analog to digital converters as follows:

With no dither, each analog input voltage is assigned one and only one code. Thus, there is no difference in the output for voltages located on the same ‘‘step’’ of the ADC’s ‘‘staircase’’ transfer curve. With dither, each analog input voltage is assigned a probability distribution for being in one of several digital codes. Now, different voltages with-in the same ‘‘step’’ of the original ADC transfer function are assigned different probability distributions. Thus, one can see how the resolution of an ADC can be improved to below an LSB.

In actual film, I presume the random inconsistencies of the individual silver halide grains is the noise source, and when watching such a film, I presume the eyes are doing the averaging through persistence of vision[2].

In either case, a key point is that you can't bring back any details by adding noise after the fact.

[1]: https://www.ti.com/lit/an/snoa232/snoa232.pdf section 3.0 - Dither

[2]: https://en.wikipedia.org/wiki/Persistence_of_vision

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#127

Post by plorkyeran »

jrm4 wrote: post #5

Yup, yet another example of the thing I'll never stop finding fascinating: ANY noticeable percieved "flaw" in any creative media will eventually become an aesthetic choice.

Even if you ignore all of the emotional attachment aspects, I think you would expect this to happen simply because people who are good at their craft will learn how to work with the shortcomings of their tools. If your cameras unavoidably add grain to everything you shoot you'll find ways to make that look good, and some of the things you come up with might not work without grain.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#128

Post by UltraSane »

supertrope wrote: post #75
UltraSane wrote: post #52

Earlier quoted context omitted.

Film grain and 24fps are both examples of people being far too attached to the technical limitations of film.

23.976 fps has been put on a pedestal as the "correct" look. Just look at the reaction to The Hobbit. However it does provide some objective advantages. 60 fps requires more lighting. Adding more lights means more electric setup and heat for actors in heavy makeup and costume. In post production that's more frames to edit.

Isn't it strange how people accepted much bigger changes like sound and color but are now balking at higher frame rates?

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#129

Post by kuschku »

Wowfunhappy wrote: post #101
crazygringo wrote: post #97

Earlier quoted context omitted.

> my real eyes don’t have grain. They definitely do at night when it's dark out. There's a kind of "sparkling" or "static" that comes in faint light. Fortunately, our eyes have way better sensitivity than cameras. But the "realism" just comes from how it was captured using the technology of the day. It's no different from phonograph hiss or the way a CRT signal blurs. The idea is to be "real" to the technology that t…

Your Van Gogh analogy makes sense for old movies. It doesn't quite explain why we're still adding grain to new movies, except for those few which are purposefully evoking older movies.

Even modern cameras have grain. If you need to integrate your scene with motion graphics, background replacement, or vfx, you'll need to remove grain on part of the image, edit it, add the original grain back where possible and synthesize new grain elsewhere.

Often it can also make sense to modify the grain for aesthetics. Denoising usually produces a less detailed result, but what you can do is denoise only the color channels, not the brightness channel. Brightness noise looks normal to us, while color noise tends to look very artificial. But by keeping the brightness noise, you avoid losing detail to the denoiser.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#130

Post by godelski »

kderbe wrote: post #107
crazygringo wrote: post #96

This fails to acknowledge that synthesized noise can lack the detail and information in the original noise . When you watch a high-quality encode that includes the actual noise, there is a startling increase in resolution from seeing a still to seeing the video. The noise is effectively dancing over a signal, and at 24 fps the signal is still perfectly clear behind it. Whereas if you lossily encode a still that disca…

Grain is independent frame-to-frame. It doesn't move with the objects in the scene (unless the video's already been encoded strangely). So long as the synthesized noise doesn't have an obvious temporal pattern, comparing stills should be fine. Regarding aesthetics, I don't think AV1 synthesized grain takes into account the size of the grains in the source video, so chunky grain from an old film source, with its big s…

People often assume noise is normal and IID but it usually isn't. It's s fine approximation but isn't the same thing, which is what the parent is discussing.

Here's an example that might help you intuit why this is true.

Let's suppose you have a digital camera and walk towards a radiation source and then away. Each radioactive particle that hits the CCD causes it to over saturate, creating visible noise in the image. The noise it introduces is random (Poisson) but your movement isn't.

Now think about how noise is introduced. There's a lot of ways actually, but I'm sure this thought exercise will reveal to you how some cause noise across frames to be dependent. Maybe as a first thought, think about from sitting on a shelf degrading.

Post reply on HN