Found something similar yesterday, which is basically deepfake for avatars - https://github.com/alievk/avatarify
Open-Source Virtual Background
11–20 of 85 posts
Re: Open-Source Virtual Background
#12I'm working on something very similar right now, so it's great to see that this is actually possible with currently available open-source resources. I'm trying to re-create Skype's background blur feature, but that I can use in OBS (obsproject.com) so I can apply it just to my webcam without messing up the other stream elements for online lectures.
I've been using the the DeepLab model from tensorflow research repo to do segmentation, but BodyPix seems even better for the job. If it performs any better, this might be the break I was looking for...
Re: Open-Source Virtual Background
#13Really good write up of a flexable approch.
If you just want an easy greenscreen https://obsproject.com/ has a very good chromakey filter and a V4L2loopback plugin https://github.com/CatxFish/obs-v4l2sink
Re: Open-Source Virtual Background
#14I get it is cool and all but, I still think that a couple of depron sheets and a can of green spaypaint gives better results.
Re: Open-Source Virtual Background
#15Found something similar yesterday, which is basically deepfake for avatars - https://github.com/alievk/avatarify
Nice. How could I get this to run on Windows?
Re: Open-Source Virtual Background
#16Since it looks like your webcam is mounted to a stationary PC and you're only really writing this for yourself, wouldn't it be a lot easier to just subtract out a static picture of your background from the feed?
Re: Open-Source Virtual Background
#17Fascinating write-up Ben, who would have known that you were a genius with image processing as well as running containers :-) Love the gory details and I didn't know about pyfakewebcam either.
Do you have a live video recorded showing how quickly it can process a stream?
Re: Open-Source Virtual Background
#18Good article, thanks. BTW, this is the method Jitsi-meet uses. They also use BodyPix. https://github.com/jitsi/jitsi-meet/blob/master/react/featur...
Re: Open-Source Virtual Background
#19Since it looks like your webcam is mounted to a stationary PC and you're only really writing this for yourself, wouldn't it be a lot easier to just subtract out a static picture of your background from the feed?
This breaks down in daylight (brightness shifts due to sun moving/clouds) or whenever anything shifts ever so slightly. Even shadows may make things appear where they shouldn't.
Chroma keying is the stable version of this idea: the single background color of a separately and well lit background removes these issues.
Re: Open-Source Virtual Background
#20Since it looks like your webcam is mounted to a stationary PC and you're only really writing this for yourself, wouldn't it be a lot easier to just subtract out a static picture of your background from the feed?
This breaks down in daylight (brightness shifts due to sun moving/clouds) or whenever anything shifts ever so slightly. Even shadows may make things appear where they shouldn't. Chroma keying is the stable version of this idea: the single background color of a separately and well lit background removes these issues.
This
I did actually tinker with this and other approaches a little.
Also the things in the background get moved around pretty frequently, e.g. what is on the couch. I don't have a dedicated office room