Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Unscreen – Remove Video and GIF Backgrounds

unscreen.com

Post reply on HN
Print view View original

11–20 of 43 posts

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#11

Post by BubRoss »

This is an area of research that has been going on for years now, called "natural image matting".

There are dozens of techniques of varying success that have been developed over the course of a decade and a half. My guess is that this is taking some more common implementation like 'closed form matting' and putting it on a server with ffmpeg. To guess the foreground I would use motion vectors as a starting point.

Also note that an alpha channel doesn't get you all the way there. You have to solve the full matting equation to extract both the foreground and alpha. You can see a bright edge around the hair in the example. The result they show still looks pretty good in general though.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#12

Post by martin-adams »

pimlottc wrote: post #8

This is neat, but I'm confused why the two sides of the split screen samples don't line up exactly. Why does removing the background shift the image?

I think they are only shifted in time which could be a quirk if the video compression.

I once had a clip that I trimmed off another scene. Only after converting the video file did a frame of that scene come back.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#13

Post by tenryuu »

a_t48 wrote: post #10
Budabellly wrote: post #9

This would make for an excellent after effects (or any post-production software for that matter) plugin. Would highly recommend going this route for the "Unscreen Pro" version that is yet to be released. Separately, because I'd bet the makers are reading, are there any plans to offer the segmentation models or APIs locally? Was looking for this for the remove.bg product as well.

It would also make an excellent OBS plugin for streamers, if such a thing doesn't already exist.

For webcams? You can use xsplit vcam. https://www.xsplit.com/ja/vcam

It uses whatever AI systems they made to single out the foreground objects from the background objects. And then it's basically just taking the camera input, applying filters or transparency and outputting it as a new video device.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#17

Post by dannyw »

BubRoss wrote: post #11

This is an area of research that has been going on for years now, called "natural image matting". There are dozens of techniques of varying success that have been developed over the course of a decade and a half. My guess is that this is taking some more common implementation like 'closed form matting' and putting it on a server with ffmpeg. To guess the foreground I would use motion vectors as a starting point. Also…

Pretty sure it's a machine learning model for video segmentation. It doesn't guess the foreground by motion: it guess it with millions of human-annotated masks.

Deep learning is making decades of research obsolete by delivering better results with more generalisation and less time.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#18

Post by atoav »

As a freelance VFX-artist the thing that annoys me about examples like these is that they don't honestly shou you the edge cases where things start falling apart (and they usually do).

A guy with dark hair in front of a white wall? I could luma key that in 10 seconds. The book example is more interesting, but there you can already see a bit of chatter (which might have to do with compression and noise tho).

In your defense you probably aim at a different target audience than people like me.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#20

Post by jtvjan »

From the submission title I assumed this was some kind of plug-in to remove those auto-playing video backgrounds from web pages. This could be a very useful tool. It sent me a 157 MB APNG in about a second, I don't even get those kinds of speeds from local file servers.
Post reply on HN