Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Unscreen – Remove Video and GIF Backgrounds

unscreen.com

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#31

Post by superasn »

julvo wrote: post #27
amerine wrote: post #3

Nice! How are you doing it?

My guess would be U-Net-like ConvNets, trained on images annotated with foreground/background segmentations. Probably with all kinds of tricks like multi-scale inference etc. However, simple frame-by-frame segmentation will probably not be enough to get temporal consistency, so for each frame's segmentation they probably take previous and following frames into account.

That is incredibly insightful. For someone having no knowledge about this field, where would one start if he wants to remove the background from images using programming?

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#35

Post by mattbee »

jtvjan wrote: post #20

From the submission title I assumed this was some kind of plug-in to remove those auto-playing video backgrounds from web pages. This could be a very useful tool. It sent me a 157 MB APNG in about a second, I don't even get those kinds of speeds from local file servers.

Yes! That's what I thought too, and I was disappointed :)

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#36

Post by numpad0 »

dillonmckay wrote: post #30
numpad0 wrote: post #26

Earlier quoted context omitted.

Mandalorian solved it by completely replacing all lighting with an LED videowall room showing camera tracked cubemap texture fed realtime from Unreal

I think the poor man’s version of this would be a fixed camera position and using a projector from the rear of the screen.

Or a known zebra background with Phillips Hue from all directions

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#37

Post by eternalny1 »

AnonC wrote: post #15

The privacy policy says that the uploaded video is deleted immediately after processing, but I'd still prefer a locally installed application for this (without any Internet access for it).

They don't mention what kind of computing resources are required for this.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#38

Post by BubRoss »

dannyw wrote: post #17
BubRoss wrote: post #11

This is an area of research that has been going on for years now, called "natural image matting". There are dozens of techniques of varying success that have been developed over the course of a decade and a half. My guess is that this is taking some more common implementation like 'closed form matting' and putting it on a server with ffmpeg. To guess the foreground I would use motion vectors as a starting point. Also…

Pretty sure it's a machine learning model for video segmentation. It doesn't guess the foreground by motion: it guess it with millions of human-annotated masks. Deep learning is making decades of research obsolete by delivering better results with more generalisation and less time.

Different techniques don't mean it isn't still natural image matting. I was guessing to give people a starting point on what to look at. Does it reference a paper somewhere? Just saying 'deep learning's doesn't really explain much.

Re: Show HN: Unscreen – Remove Video and GIF Backgrounds

#39

Post by julvo »

superasn wrote: post #31
julvo wrote: post #27

Earlier quoted context omitted.

My guess would be U-Net-like ConvNets, trained on images annotated with foreground/background segmentations. Probably with all kinds of tricks like multi-scale inference etc. However, simple frame-by-frame segmentation will probably not be enough to get temporal consistency, so for each frame's segmentation they probably take previous and following frames into account.

That is incredibly insightful. For someone having no knowledge about this field, where would one start if he wants to remove the background from images using programming?

Depending on the type of image, a simple solution could be using OpenCV and some clever heuristics.

For a deep learning approach, I would start by looking into literature on semantic segmentation. Here is a blog post I just found which gives an intro: [1]

With state-of-the-art models (e.g. DeepLabV3) and a good dataset of foreground/background segmentations, the results could be of useful quality already.

The next step would be to look into literature on image matting (e.g. deep image matting [2]) which instead of trying to classify each pixel as foreground/background, regresses the foreground colour and transparency.

___

[1] https://divamgupta.com/image-segmentation/2019/06/06/deep-le...

[2] https://arxiv.org/abs/1703.03872

Post reply on HN