Live data from Hacker News

Background Matting: The World is Your Green Screen

grail.cs.washington.edu

21–30 of 60 posts

Re: Background Matting: The World is Your Green Screen

#21
post #13

What's novel about this? If you have a background picture, you have all the info you need to identify your subject - just plain subtraction. I think this is what the Photo Booth app on my circa-2012 MacBook does, quite effectively.

This new method works well for partially transparent regions (hair) and allows slightly larger background movement and color overlap between foreground and background.

Re: Background Matting: The World is Your Green Screen

#22

Peripheral: What is the benefit of halving these artificial backgrounds? Apart from „it’s fun“, which wears off after about a minute? In my experience (Zoom Meetings), there’s blurring/artefacts around the edge of the head and the image quality seems to suffer as well. I had a meeting where one participant uses an actual green screen, and the difference was remarkable, with none of the issues above.

There are two parts to background matting. The first is removing the existing background and the second is replacing it with something else. Removing the background improves the focus on the foreground - people watching can see you better and they'll listen more closely because they're not distracted by what's behind you. The second part, replacing the background with something else, might be done because you don't want people to see where you are, or because you want to overlay your foreground video on a presentation. Being able to pretend you're on a holodeck or a desert island is a trivial use of the tech.

Re: Background Matting: The World is Your Green Screen

#23
post #2

Is it fast enough to operate on a live video feed from, say, a webcam?

Yes. OBS offers it. Teams and Zoom have guesswork-based imitations.

The Zoom guesswork-based imitation is pretty good, and it seems to be optimized for a single person's movement. It gets confused when there are more actors like a child or dog entering from stage left.

Re: Background Matting: The World is Your Green Screen

#25
post #13

What's novel about this? If you have a background picture, you have all the info you need to identify your subject - just plain subtraction. I think this is what the Photo Booth app on my circa-2012 MacBook does, quite effectively.

Ex-VFX person here.

Green screens are crap with hair, because its translucent the green/blue bleeds through which means that it has to be cleaned up by hand.

Then there are the situations where there isn't a green screen. Again manual cleanup is required. Each frame needs to be cut out by hand. 24 times a second.

The same with a difference matte. Cameras are noisy, so there is constant noise in the alpha channel. This makes the effect look wobbly and cheap.

What this method does is pull a key from a difference matte, and makes it look good.

Re: Background Matting: The World is Your Green Screen

#26
post #13

What's novel about this? If you have a background picture, you have all the info you need to identify your subject - just plain subtraction. I think this is what the Photo Booth app on my circa-2012 MacBook does, quite effectively.

This is a question we’ve gotten quite a bit (second author here). A good intuition is that if it were easy to do it already with any background, professional studios wouldn’t be spending so much money on green screens. Background subtraction is pretty poor in general without very constrained setups. Our goal is really to provide professional quality without any of the equipment.

And can your solution do what studios want, namely process a 4K-video artifact free, when played back on a cinema screen? I doesn't look like that tbh if I watch the second video ("Ours real" is your work?).

And yeah, it requires constrained setup and a lot of additional work, because even before you "subtract" the background you have to think about lighting (your demo video might have very nice background matting, but the lighting is off, so it's relatively useless except for toy applications (which there are a lot).

Also: did you compare somewhere withe the very basic fixed-exposure method? Beause for fixed exposure, background and camera placement I suppose this should work just as well... Still I think this is a really cool project, I didn't get disappointed like with the last link of that sort, where someone tried the same with horrible artifacting.

Re: Background Matting: The World is Your Green Screen

#27
This does look like an interesting step forward however I’ve found the biggest limitations of previous techniques to be

- people who are using occupying a wider z-axis (for example leaning forwards in the camera or who have arms in front of them)

- people holding objects like cups

How well do your method handle those kind of situations?

Re: Background Matting: The World is Your Green Screen

#28
post #21
post #13

What's novel about this? If you have a background picture, you have all the info you need to identify your subject - just plain subtraction. I think this is what the Photo Booth app on my circa-2012 MacBook does, quite effectively.

This new method works well for partially transparent regions (hair) and allows slightly larger background movement and color overlap between foreground and background.

I think they baited you with their "loook, movement" replacement videos. As far as I can tell, their inputs have a fixed background and are of constant exposure and camera position.

Re: Background Matting: The World is Your Green Screen

#29
post #13

What's novel about this? If you have a background picture, you have all the info you need to identify your subject - just plain subtraction. I think this is what the Photo Booth app on my circa-2012 MacBook does, quite effectively.

I thought the same thing. I remember vividly playing with PhotoBoot on my aunt's MacBook, I really enjoyed the rollercoaster!

Re: Background Matting: The World is Your Green Screen

#30
post #11

Earlier quoted context omitted.

https://github.com/obsproject/obs-studio

I found the chroma key implementation, but this seems to assume you have a green screen or similar: https://github.com/obsproject/obs-studio/blob/master/plugins... I thought GP was asking for an implementation of background removal without a green screen.

Indeed. I'm fairly sure it's not in OBS. Which is why I asked for a link to where it was implemented. Surprisingly I didn't receive one...
Post reply on HN