Open-Source Virtual Background • Hacker News Board

Re: Open-Source Virtual Background

#51

Post by rkeene2 » Fri, Apr 10, 2020, 1:35 PM UTC

wdfx wrote: post #48

Using video loopback opens up some great creative possibilities for fun with video conferencing. There's one thing I'd love to achieve though, which seems not possible on Linux desktop (specifically kubuntu).... I want to be able to use the loopback as the source for screen share instead of webcam; i.e. to use the loopback as the conference presentation. Has anyone got any ideas how to achieve this? Given most confer…

The most naive solution would be to run a nested framebuffer with just the meeting and the windows you would like to share.

Re: Open-Source Virtual Background

#52

Post by taborj » Fri, Apr 10, 2020, 1:55 PM UTC

soylentcola wrote: post #44

I'd love to see something like this as a plugin for OBS. I've been using it lately due to all the video conferencing we're all growing to love since it's got basic color correction/manual controls for my webcam feed. It's got the option for "real" chromakey, but like the author, I don't have a green screen, Amazon isn't scheduling any deliveries for another month, and I don't feel like a trip to the fabric store woul…

I agree, a plugin like this for OBS would be amazing.

I, too, have been playing with the chromakey in OBS, and like you tried a bunch of different sheets and blankets. The one that worked the best I discovered by accident; my wife was pulling out a puzzle, and with it she pulled out a green felt puzzle mat (really just a large rectangle of green felt). It was perfect. But I had to get it from her so I could use it on my meeting before she started her puzzle. After some brief, tense negotiating, I was able to snag it. All I have to do is the dishes for another month!

It works really well, but this would be way better and easier to set up.

Re: Open-Source Virtual Background

#53

Post by dhimes » Fri, Apr 10, 2020, 1:56 PM UTC

dh2424 wrote: post #49

This is pretty awesome work, but I just wanted to point out that Zoom doesn't actually require a green screen. If you uncheck the "I have a green screen" button, choosing your own virtual background still looks really good, although of course you're not going to get any crazy effects like this script adds.

Depends on your hardware. I have a Win machine, but need a green screen because I don't have the horsepower.

Re: Open-Source Virtual Background

#54

Post by taborj » Fri, Apr 10, 2020, 1:58 PM UTC

Karupan wrote: post #34

monkeydust wrote: post #15

Earlier quoted context omitted.

Nice. How could I get this to run on Windows?

I'm guessing you can use something like [1] to create a virtual device on Windows [1] https://webcamoid.github.io

I've found the VirtualCam plugin for OBS works better, but YMMV

Re: Open-Source Virtual Background

#55

Post by taborj » Fri, Apr 10, 2020, 1:59 PM UTC

franga2000 wrote: post #12

I'm working on something very similar right now, so it's great to see that this is actually possible with currently available open-source resources. I'm trying to re-create Skype's background blur feature, but that I can use in OBS (obsproject.com) so I can apply it just to my webcam without messing up the other stream elements for online lectures. I've been using the the DeepLab model from tensorflow research repo t…

I am eagerly anticipating this. I use OBS and chromakey to have some fun on my MS Teams meetings, but it's a hassle to set up the green screen every time I want to be a bit cheeky.

Do you have a github/gitlab/something else link?

Re: Open-Source Virtual Background

#56

Post by LeifCarrotson » Fri, Apr 10, 2020, 2:09 PM UTC

dhimes wrote: post #53

dh2424 wrote: post #49

This is pretty awesome work, but I just wanted to point out that Zoom doesn't actually require a green screen. If you uncheck the "I have a green screen" button, choosing your own virtual background still looks really good, although of course you're not going to get any crazy effects like this script adds.

Depends on your hardware. I have a Win machine, but need a green screen because I don't have the horsepower.

And your software. I have a dual-booting machine, the green screen is required in Linux but not in Windows.

Re: Open-Source Virtual Background

#57

Post by jonfw » Fri, Apr 10, 2020, 2:25 PM UTC

chpmrc wrote: post #46

Is there something Python can't do? Can't wait for a React/Vue like framework for Python.

It can't run in the browser which makes this a non-starter.

It also can't make efficient use of resources, which explains it's lack of prevalence in embedded/back-end apps

Re: Open-Source Virtual Background

#58

Post by jamieweb » Fri, Apr 10, 2020, 2:26 PM UTC

peer2pay wrote: post #16

Since it looks like your webcam is mounted to a stationary PC and you're only really writing this for yourself, wouldn't it be a lot easier to just subtract out a static picture of your background from the feed?

This is exactly how it worked in Photo Booth on OS X over 10 years ago. It was good enough for short clips, but not reliable for long sessions.

Random example clip: https://youtu.be/T5uqB1Kqukw

Re: Open-Source Virtual Background

#59

Post by manbash » Fri, Apr 10, 2020, 2:27 PM UTC

Great read! I am curious then why the Linux client doesn't supports this, if all it takes is to send out our webcam stream data to be processed server-side?

P.S. What happens when they do e2ee on the webcam stream?

Re: Open-Source Virtual Background

#60

Post by morsch » Fri, Apr 10, 2020, 2:34 PM UTC

wdfx wrote: post #48

Using video loopback opens up some great creative possibilities for fun with video conferencing. There's one thing I'd love to achieve though, which seems not possible on Linux desktop (specifically kubuntu).... I want to be able to use the loopback as the source for screen share instead of webcam; i.e. to use the loopback as the conference presentation. Has anyone got any ideas how to achieve this? Given most confer…

Has anyone got any ideas how to achieve this?

Yes. Thanks for giving me a reason to write this up.

1. Download and install OBS. OBS will be your video processor; among other things it's super easy to make it capture the whole screen or individual windows.

2. Install the v4l2 loopback kernel module[1]. This makes it possible to have a virtual webcam. On Ubuntu 19.10, this was as easy as apt install v4l2loopback-dkms and then modprobe v4l2loopback.

3. Install the OBS plugin obs-v4l2sink[5]. This exports the OBS output to the new virtual webcam device. I just installed the deb file provided by the project[2]. In OBS, under Tools, select v4l2sink and Start.

That's all I had to do. Surprisingly straightforward. At least Chrome and Firefox[3] will now pick up a "Dummy Video Device" webcam that streams the window, or whatever scene I set up in OBS.

In my case, the primary advantage was that this virtual webcam is streamed in Jitsi Meet at a higher quality/framerate than the regular desktop share feature. It's also much lower latency than both Twitch and Youtube Live streaming (Jitsi Meet/WebRTC: You also get to enjoy the rich feature set OBS provides for Twitch streams; for one thing, you can include the real webcam video.

Bonus: Desktop audio "just worked" in Firefox, which offers the pulseaudio monitor (loopback) device as an input. Chrome doesn't -- probably the intended behaviour. I'm sure there's a workaround.

[1] https://github.com/umlaeute/v4l2loopback

[2] https://github.com/CatxFish/obs-v4l2sink/releases

[3] For some reason, Gnome's Cheese won't

[4] Microsoft's Mixer allegedly has super-low-latency streaming (FTL protocol), but new account are cleared manually and I haven't had the chance to try

[5] For Windows, you can use OBS Virtualcam https://obsproject.com/forum/resources/obs-virtualcam.539/