Live data from Hacker News

It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

youtube.com

81–90 of 119 posts

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#81

Anyone tried using lidar and just cut/measure distance to the object?

Apparently they used something similar for production on avatar: stereo cameras for depth estimation which allowed realtime depth composition of CG characters onto the shots they were taking, which makes it a lot easier to get everyone on the same page about the scene, especially with characters that are outside normal human proportions. But it wasn't good enough for the final shots.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#82
post #10

Earlier quoted context omitted.

With regards to the sodium vapor process, an idea has been percolating in the back of my head ever since I saw that video. But I don't really have the budget to try it out. theory: make the mask out of non-visable light illuminate the backing screen in near Infra-Red light. (after a bit of thought I chose near-IR as opposed to near-UV for hopefully obvious reasons) point two cameras at a splitting prism with a near I…

I'll do you one better, which requires no special cameras (most have IR filters) nor double cameras or prisms. Shoot the scene in 48 or 96 fps. Sync the set lighting to odd frames. Every odd frame, the set lights are on. Every even frame, set lights are off. For the backing screen, do the reverse. Even frames, the backing screen is on. Odd frames, backing screen is off. There you go. Mask / normal shot / Mask normal…

Somebody recently used a variation of this to get good video of welding - basically a camera synced with a very bright (strobe-ish) light, brighter than the weld itself, so you adjust the camera to the ludicrous-but-consistent brightness level and get details of the weld and the surroundings. https://www.youtube.com/watch?v=wSUxK8q4D0Q (Chronos "Helios", from early 2025)

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#83
I use the Adobe version of this in Photoshop every day and I assumed that Adobe solved this the same way, but used professionals to cut out the subjects from the backgrounds then fed both versions into their AI.

Since they added it a year or so ago it has been game-changing. I'm cutting out portraits every day and having a magical tool that cuts out the subject with perfect hair cut out with a single click is sci-fi.

Here's a demo of Photoshop's tool:

https://www.youtube.com/watch?v=SNVJN6PKeGQ

(the other magical Photoshop tool is the one that removes reflections from windows, which is even more insane when you reverse it and tell it you only want the reflection and not what's on the other side of the glass)

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#84
post #61
post #32

Earlier quoted context omitted.

Incandescent and fluorescent lights already flicker at your AC power frequency. Just gotta be higher than that

No. Incandescent lights flicker at twice your AC power frequency -- to a decent approximation, their power is proportional to V^2. But this is input power -- the cooling of the filament is slowish and the modulation depth is low. Most people aren't bothered by this. Fluorescent lights with old or very crappy "magnetic" ballasts flicker at twice the mains frequency, with deep modulation. The effect on people varies fr…

Hey thanks for clearing this up. I had no idea that CFLs and fluorescent lights with electronic ballasts now flicker at ~ 20kHz.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#85
post #9

I'm a software engineer that, like the vast majority of you, uses AI/agents in my workflow every day. That being said, I have to admit that it feels a little weird to hear someone who does not write code say that they built something, without even mentioning that they had an agent build it (unless I missed that).

Is a carpenter who relied on a CNC to cut all of their pieces a builder?

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#86
post #23

There's still a bug: the glass with water does not distort the checker pattern in the background at 24:12.

Good spot! That is the product working as intended though. The background doesn't exist except as an asset that replaces the green screen. The tool is meant to replace the green screen without the need for manual rotoscoping. Even in a traditional process, the distortion needs to be done by VFX as a separate process. To do that though, they still need the green screen keyed out and this tool does that.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#87
post #60

As far as alternatives, I wonder if anyone has tried a screen that cycles through colors in a known sequence. Using this modulating-color screen, it might actually be easier to separate the subject because you get around the "green shirt over green screen" problem. You might even be able to use a time sampling to correct the light cast on the subject from the screen as you would have a full spectrum of response. I co…

the general problem with any technique that isnt just throw some vaugely green thing behind our actors is that setting up complicated tech like this on an actual film set is extremely expensive. both the time it would take and the risk of it not working. so you end up with a dedicated permanent stage install but now you need to get the actors and crew to that place. better keys isnt a bad enough problem to justify that effort/cost. even the highly touted "virtual production" mandalorian stuff where you just put a big led wall behind the actors has shown to be more expensive than traditional vfx unless you tightly control the creative or approach.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#89
post #47

Earlier quoted context omitted.

Yes. But the main issue is in the way they formulate the problem. Their output is always a transparency mask, which of course will never handle distortions.

Right. Things like this are why it’s difficult integrating AI into professional movie pipelines— they’re super complex in ways AI cannot (yet) replicate for very good reasons that seem superfluous or trivially replaceable by people not familiar with them.

People in ML have this kind of belief rooted in the bitter lesson, that everything will eventually sort itself out given enough scale and data. That often makes them ignore the nuances of particular problem domains. CC is the opposite of that, it's just impossible to do everything at once.

Re: It Took Me 30 Years to Solve This VFX Problem – Green Screen Problem [video]

#90
post #47

Earlier quoted context omitted.

>there’s no way that’s going to happen automatically They train their model in a pretty straightforward way, it can also be used to capture the distortion as well, just use a non-monochrome (possibly moving) background optimized for this. It's a matter of effort and attention to detail during training (uneven green screen lighting, reflections, etc), not fundamental impossibility

Yes. But the main issue is in the way they formulate the problem. Their output is always a transparency mask, which of course will never handle distortions.

youd have to train it to also generate and st map of the distortions but creating the ground truth version of that from the synthetic data would add a lot more to render. also its very easy to plausibly fake, its not something humans are good at seeing and knowing its wrong. you can tell its completely missing but accurate vs just distorted in a plausible way is not something most brains are tuned to notice.
Post reply on HN