51–60 of 60 posts
Very cool. Next step would be to emulate lighting in the target scene, but that probably requires pose detection and facial landmarks for accurate shading.
This does look like an interesting step forward however I’ve found the biggest limitations of previous techniques to be - people who are using occupying a wider z-axis (for example leaning forwards in the camera or who have arms in front of them) - people holding objects like cups How well do your method handle those kind of situations?
The neural network used seems mostly for allowing for variations in lighting and dealing with the fuzzy effects you'll usually get around the subject. Depth of the subject doesn't appear to be relevant here.
This does look like an interesting step forward however I’ve found the biggest limitations of previous techniques to be - people who are using occupying a wider z-axis (for example leaning forwards in the camera or who have arms in front of them) - people holding objects like cups How well do your method handle those kind of situations?
I'm not sure how either of those situations would trip up the system they're using. For a system trained on the background image, what difference does it make if the subject is holding a cup? The cup is not the background image, and it would be obvious in the same way that it's obvious the subject isn't the background image.
Anybody know of any work done to improve greenscreen keying? The current old school techniques work quite poorly and require so much manual work. I would imagine with the new work coming out with neural nets etc. there would be possibilities for improvement. This is very cool work and good for certain applications, but seems to produce similar problems like greenscreen to some edges.
Very cool. Next step would be to emulate lighting in the target scene, but that probably requires pose detection and facial landmarks for accurate shading.
I have been thinking that a killer zoom product would be a USB controller for an rgb led strip that helps match color with your virtual background.
Peripheral: What is the benefit of halving these artificial backgrounds? Apart from „it’s fun“, which wears off after about a minute? In my experience (Zoom Meetings), there’s blurring/artefacts around the edge of the head and the image quality seems to suffer as well. I had a meeting where one participant uses an actual green screen, and the difference was remarkable, with none of the issues above.
I've seen someone advised that the background on their webcam makes them "look poor", where the concern was that looking poor is a (perverse) impediment to getting paid work, but they can't exactly move, especially under lockdown. It may be better to use a calm artificial background in that case.
See also people doing online-conference presentations and Youtube videos. I've seen quite a few of those are using virtual backgrounds.
Perhaps for the same reason - thousands of people may see the video, and some people, having made the effort to put on a nice suit/makeup/etc, get a haircut, and look their best, don't want thousands of people to see what their not so nice home looks like behind it.
Anybody know of any work done to improve greenscreen keying? The current old school techniques work quite poorly and require so much manual work. I would imagine with the new work coming out with neural nets etc. there would be possibilities for improvement. This is very cool work and good for certain applications, but seems to produce similar problems like greenscreen to some edges.
Modern green screen plugins/filters are miles better than what they used to be to the point that if the keying is hard, then the image must not have been produced well. By that, I mean an evenly list background (no light fall off producing gradients). Proper lighting of the subject. Proper distance from the background (helps reduce the edges and color tinting).
Earlier quoted context omitted.
Ex-VFX person here. Green screens are crap with hair, because its translucent the green/blue bleeds through which means that it has to be cleaned up by hand. Then there are the situations where there isn't a green screen. Again manual cleanup is required. Each frame needs to be cut out by hand. 24 times a second. The same with a difference matte. Cameras are noisy, so there is constant noise in the alpha channel. Thi…
Ex-VFX part-time software team member here. I can assure you they read these papers. If it is good, it will be part of a future version of the software.
The foundry have been trying to get this into Nuke for years. The problem is that normally you get flickering, as you'll have seen.
Earlier quoted context omitted.
Ex-VFX person here. Green screens are crap with hair, because its translucent the green/blue bleeds through which means that it has to be cleaned up by hand. Then there are the situations where there isn't a green screen. Again manual cleanup is required. Each frame needs to be cut out by hand. 24 times a second. The same with a difference matte. Cameras are noisy, so there is constant noise in the alpha channel. Thi…
Are there no tools that automate the cleanup of green-screen artifacts?
The originally submitted URL ( https://www.catalyzex.com/paper/arxiv:2004.00626 ) points to https://github.com/senguptaumd/Background-Matting , which points to https://grail.cs.washington.edu/projects/background-matting , which points to https://arxiv.org/abs/2004.00626 , which points to https://arxiv.org/pdf/2004.00626.pdf , which is inlined at the originally submitted URL. I'm not sure what's going on here, but on…
The link share here was likely with keeping the relevance of this project to HN in mind, and that easy access to the code and authors would be valuable for anyone here looking to take it further.
Thanks for clarifying the convention here on HN, being transparent, and for updating accordingly. Much appreciated.
Always open to feedback if you have any as well! :)