Live data from Hacker News

AV1@Scale: Film Grain Synthesis, The Awakening

netflixtechblog.com

81–90 of 230 posts

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#81

When you talk on a cellphone the codec AMR-WB nominally captures 50 Hz - 7000 Hz. However that's only on the optional highest bitrate 23.85 Kbps. The most common bitrate 12.65 Kbps only goes up to 6400 Hz and synthesizes 6400 - 7000 Hz from lower frequencies and noise as it sounds better than not having the noise!

This sent me down a very interesting rabbit hole, thanks!

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#82

happier and happier about leaving behind digital media to return to physical. to me this is literally slop. i want the uncompressed file stop selling me stepped on product

Your statement makes me think that one of the following is true:

1. You prefer Betamax or VHS to digital media (highly unlikely)

2. You own laserdiscs (limited to 480i)

3. You own 35mm prints of film.

Since all other formats film has been made available on are both digital media and compressed.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#83

Earlier quoted context omitted.

> provide distributors with a noiseless source plus grain synthesis parameters. What parameters would that be? Make it look like Eastman Ektachrome High-Speed Daylight Film 7251 400D? For years, people have taken film negative onto telecines and created content of grain to be used as overlays. For years, colorists have come up with ways of simulating the color of specific film stocks by using reference film with test…

And yet here we are: DNR -> fancy grain -> DNR -> basic, approximated grain. Because noise doesn’t compress. And you get compression artifacts even in Blu-ray releases. What’s the point of applying fancy grain when what a lot viewers end up seeing is an ugly smudge?

Because it looks amazing in the editing studio. Just like the sound mix is incredible on the Atmos monitors in the sound mixing room, even though the home viewers have a soundbar at best and tiny stereo speakers in a flat panel typically. The dynamics and dialog channel will be fucked. But that’s user error.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#84

The original grain that is captured is actually a detail and not total random noise. I believe you can make up the vague sense of original scene if you could somehow extract that grain/noise alone. It's like reducing an image to tiny dots with dithering (reminds of Atinkson dithering). Those grains are not a noise, they are a detail, actual data. That's why real grain looks good IMO.

If you extract the grain (e.g. by subtracting a blurred version of the image), the result contains mostly noise, not meaningful scene information outside of some variation according to image brightness. Film grain is random, so the extracted "grain layer" doesn’t encode original image detail but film grain itself encodes relative lightness in its _density_.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#85
post #3

There are definite philosophical questions over the merits of adding noise, but the problem with their example here is their denoising process appears to excessively blur everything, so both it and the synthesized grain image look noticeably less sharp than the source. The grain itself also looks too much like basic noise, and not really grain like.

Agree, as someone who has spent way too much time studying the way motion picture film looks up close, this isn’t very realistic looking. It’s really just a form of dithering.

Forget the film grain, give us film lint and film hair!

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#86

Earlier quoted context omitted.

And yet here we are: DNR -> fancy grain -> DNR -> basic, approximated grain. Because noise doesn’t compress. And you get compression artifacts even in Blu-ray releases. What’s the point of applying fancy grain when what a lot viewers end up seeing is an ugly smudge?

Because it looks amazing in the editing studio. Just like the sound mix is incredible on the Atmos monitors in the sound mixing room, even though the home viewers have a soundbar at best and tiny stereo speakers in a flat panel typically. The dynamics and dialog channel will be fucked. But that’s user error.

Movies are best enjoyed in the theater.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#87

> This grain, formed from tiny particles during the film’s development, is more than just a visual effect. It plays a key role in storytelling by enhancing the film’s depth and contributing to its realism. I never understood the “grain = realism” thing. my real eyes don’t have grain. I do appreciate the role of grain as an artistic tool though, so this is still cool tech

It's doubling back on itself. The film grain makes the footage look "cinematic" because it's how old movies looked.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#88

The original grain that is captured is actually a detail and not total random noise. I believe you can make up the vague sense of original scene if you could somehow extract that grain/noise alone. It's like reducing an image to tiny dots with dithering (reminds of Atinkson dithering). Those grains are not a noise, they are a detail, actual data. That's why real grain looks good IMO.

This is just a fact of lossy compression: you want to throw away information that contributes less to the perception of the video so that you can describe it with fewer bits of information.

There are two possible advantages for this kind of grain synthesis. For Netflix, they could produce the same perceived quality at lower bitrates, which reduces costs per view and allows customers with marginally slow connections to get a higher quality version. For a consumer, the advantage would be getting more non-grain detail for a fixed bitrate.

You are right that if you subtract the dentists frame from the raw one, showing only the estimated noise, you would get some impression of the scene. I think there’s two reasons for this. Firstly, the places where the denoiser produced a blurry line that should be sharp may show up as faint lines. I don’t think this is ‘hidden information’ so much as it is information lost to lossy compression. In the same way, if you look at the difference between a raw image and one with compression, you may see some emphasized edges due to compression artefacts. Secondly, the less exposed regions of the film will have more noise so noisiness becomes a proxy for darkness, allowing some reproduction of the scene. I would expect this detail to be lost after adjusting for the piecewise linear function for grain intensity at different brightness levels.

Perhaps a third thing is the level of noise in the blacks and the ‘grain size’ or other statistical properties tell you about the kind of film being used, but I think those things are captured in the film grain simulation model.

Possibly there are some other artefacts like evidence of special effects, post processing, etc.

Re: AV1@Scale: Film Grain Synthesis, The Awakening

#89
post #3

There are definite philosophical questions over the merits of adding noise, but the problem with their example here is their denoising process appears to excessively blur everything, so both it and the synthesized grain image look noticeably less sharp than the source. The grain itself also looks too much like basic noise, and not really grain like.

Agree, as someone who has spent way too much time studying the way motion picture film looks up close, this isn’t very realistic looking. It’s really just a form of dithering.

Video codecs use a lot of tricks based on human perception, perhaps it's much closer to the real thing when in motion vs a still image?
Post reply on HN