Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

youtube.com

Post reply on HN
Print view View original

101–110 of 119 posts

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#101

Post by actionfromafar »

shdudns wrote: post #74
actionfromafar wrote: post #16

Earlier quoted context omitted.

I'll do you one better, which requires no special cameras (most have IR filters) nor double cameras or prisms. Shoot the scene in 48 or 96 fps. Sync the set lighting to odd frames. Every odd frame, the set lights are on. Every even frame, set lights are off. For the backing screen, do the reverse. Even frames, the backing screen is on. Odd frames, backing screen is off. There you go. Mask / normal shot / Mask normal…

Two problems: - It'll bleed on fast motion. Hair in the wind would just not work. - Incandescent lights are out. You could solve both by having two ghost frames shot very close to the real frame (no need to evenly space the frames, after-all) and using strobing a high powered laser. You'd need very fast sensor or another one optically on the same position.

At some point higher fps solves it. Is 240 fps enough?

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#102

Post by adrian_b »

throwway120385 wrote: post #28
somat wrote: post #13

Earlier quoted context omitted.

That is far-IR, thermal stuff, Near-IR, 700 nanometer-ish is right below red in human vision. Camera sensors can pick up a little near-IR so they have have a filter to block it. If that filter was removed and a filter to block visable light was used in place you would have a camera that can only see non-visable light. Poorly, the camera was not engineered to operate in this bandwidth, but it might be good enough for…

You can actually dimly perceive near-IR LEDs -- they'll glow slightly red in darkness.

That depends on how "near" they are.

The sensitivity to red light decreases quickly at wavelengths greater than 650 nm, but light can still be perceived if it is strong enough, up to around 780 nm.

Many so-called near-IR LEDs may actually be somewhere around 750 nm, so they are still visible on a dark background, even if they are perceived as extremely dim.

On the other hand, there are many near infrared LEDs around 900 nm and those are really invisible. Near-infrared LEDs around 1300 nm or around 1550 nm are also completely invisible.

An invisible near-infrared laser beam could become visible due to double-photon absorption, but if a beam of such intensity as to cause double-photon absorption hits your retina, there are more serious things to worry about.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#103

Post by mk_stjames »

In case anyone wanted technical details of the NN, I dug into the repo:

Its a transformer, with a CNN refiner after. Specifically, a ViT using the Hiera architecture (https://github.com/facebookresearch/hiera)

The Hiera ViT has dual decoder heads, one for the alpha and one for the RGD foreground, and then a small CNN refiner network to solve some artifacting in the output from the Hiera model.

I'd be very interested to see a long form tech talk of Niko explaining his process of learning ML ropes and building this model.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#104

Post by tlb »

SoftTalker wrote: post #75
amluto wrote: post #61

Earlier quoted context omitted.

No. Incandescent lights flicker at twice your AC power frequency -- to a decent approximation, their power is proportional to V^2. But this is input power -- the cooling of the filament is slowish and the modulation depth is low. Most people aren't bothered by this. Fluorescent lights with old or very crappy "magnetic" ballasts flicker at twice the mains frequency, with deep modulation. The effect on people varies fr…

The fluorescent light strobing is why you often see fluorescent tubes in pairs. They will be wired in opposite phase to cancel the strobing.

I think the total light output of each bulb in the pair is the same at all points in time, but the orange-blue gradient is reversed. So when one is orange at one end, the bulb beside it is blue at that end.

IIRC, the end that's negative looks orange, because the electrons emitted from the filament haven't gotten up to speed yet and can't ionize the mercury atoms at that end to the highest states.

If you didn't do this, you'd see 60 Hz strobing when you looked at one end.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#105

Post by DrewADesign »

orbital-decay wrote: post #89
DrewADesign wrote: post #69

Earlier quoted context omitted.

Right. Things like this are why it’s difficult integrating AI into professional movie pipelines— they’re super complex in ways AI cannot (yet) replicate for very good reasons that seem superfluous or trivially replaceable by people not familiar with them.

People in ML have this kind of belief rooted in the bitter lesson, that everything will eventually sort itself out given enough scale and data. That often makes them ignore the nuances of particular problem domains. CC is the opposite of that, it's just impossible to do everything at once.

It’s certainly a big part of the ML scene, but to a slightly lesser extent, a cultural facet of development in general. It’s not all bad! Many people have solved problems that nobody in their right mind would have attempted knowing the nitty gritty details; often the problem they solved wasn’t the one they intended to solve, or they only solve one small subset of it, but were still valuable advancements. Unfortunately, that also leads to reinforcing some people’s Dunning-Krueger-fueled insistence that they can solve another field’s difficult problems with a few thought experiments, and the only reason it hasn’t already been solved is because nobody thought to ask a developer as smart as them to momentarily consider the problem. Non-developers in tech often bear the brunt of it: moving into design after a decade of dev work, that irritating mindset was one of the reasons I left tech altogether a couple years later.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#106

Post by nkrisc »

amelius wrote: post #23

There's still a bug: the glass with water does not distort the checker pattern in the background at 24:12.

Not a bug, creating distortion in the comped in backgrounds is not what this tool does. It creates a transparency mask. How do you propose a transparency mask captures distortion artifacts?

That distortion to the new background would have to be added in by the artist.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#107

Post by randyrand »

somat wrote: post #10
Springtime wrote: post #3

In an earlier video they made a couple years back about Disney's sodium vapor technique Paul Debevec suggested he was considering creating a dataset using a similar premise: filming enough perfectly masked references to be able to train models to achieve better keying. So it was interesting seeing Corridor tackle this by instead using synthetic data.

With regards to the sodium vapor process, an idea has been percolating in the back of my head ever since I saw that video. But I don't really have the budget to try it out. theory: make the mask out of non-visable light illuminate the backing screen in near Infra-Red light. (after a bit of thought I chose near-IR as opposed to near-UV for hopefully obvious reasons) point two cameras at a splitting prism with a near I…

Couldn't this be summarized as the Sodium Vapor technique but with near-IR? Or do I misunderstand something?

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#108

Post by Coeur »

MrVitaliy wrote: post #62

Anyone tried using lidar and just cut/measure distance to the object?

Well there was ZCam, which was a time-of-flight add-on for ENG cameras. It couldn't really handle edges perfectly and existing bluescreen tech was good enough for TV production, so they pivoted into gaming and sold to Microsoft for the Kinect.

https://en.wikipedia.org/wiki/ZCam (Demo: https://www.youtube.com/watch?v=s7Kcmx29RCE )

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#110

Post by mcurist »

dgently7 wrote: post #87
jayd16 wrote: post #60

As far as alternatives, I wonder if anyone has tried a screen that cycles through colors in a known sequence. Using this modulating-color screen, it might actually be easier to separate the subject because you get around the "green shirt over green screen" problem. You might even be able to use a time sampling to correct the light cast on the subject from the screen as you would have a full spectrum of response. I co…

the general problem with any technique that isnt just throw some vaugely green thing behind our actors is that setting up complicated tech like this on an actual film set is extremely expensive. both the time it would take and the risk of it not working. so you end up with a dedicated permanent stage install but now you need to get the actors and crew to that place. better keys isnt a bad enough problem to justify th…

And all the people aware of the production technique watch it and imagine the characters saying "We can't run from the monster in different direction, our virtual production stage is precisely this big!"

Green screens just more flexible

Post reply on HN