Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-Source Virtual Background

elder.dev

Post reply on HN
Print view View original

31–40 of 85 posts

Re: Open-Source Virtual Background

#34

Post by Karupan »

monkeydust wrote: post #15
Karupan wrote: post #11

Found something similar yesterday, which is basically deepfake for avatars - https://github.com/alievk/avatarify

Nice. How could I get this to run on Windows?

I'm guessing you can use something like [1] to create a virtual device on Windows

[1] https://webcamoid.github.io

Re: Open-Source Virtual Background

#35

Post by TeMPOraL »

BenTheElder wrote: post #30
sigwinch28 wrote: post #25

You mention the ~10FPS performance. It seems like moving all that data backwards and forwards between Python and Node might be a bottleneck, no?

I need to profile it more closely, I actually don't remember the exact FPS etc. but I don't expect that to be the limiting factor. The inference / ml is expensive (which I did profile initially...), and I suspect not really optimized on this backend. It appears to be faster with webGL in the browser. I sorta stopped worrying about it once it was "good enough" to show up to a few meetings with, but with all the attent…

I guess I must be completely miscalibrated wrt. performance of newer technologies, because I'd imagine it's the opposite. In particular, I'd be surprised to get a Python+Node loop passing large amounts of data around like that to run 30+ FPS, unless everything Python-side is carefully written to do everything on C side. At the same time, I'd assume the inference/ML part is the fastest one, because, as far as I understand how NNs work, they're supposed to be blazingly fast once trained (it's just lots of parallelizable linear algebra). Is the inference part in your solution doing anything more complicated than that in real-time?

Re: Open-Source Virtual Background

#36

Post by benoliver999 »

shrx wrote: post #33
knotty66 wrote: post #18

Good article, thanks. BTW, this is the method Jitsi-meet uses. They also use BodyPix. https://github.com/jitsi/jitsi-meet/blob/master/react/featur...

Where is the virtual background option in Jitsi? I couldn't find it.

On Jitsi Meet, in the browser, there is a menu on the bottom right. Click it and there are loads of options there including blur background.

Re: Open-Source Virtual Background

#38

Post by porjo »

benoliver999 wrote: post #36
shrx wrote: post #33

Earlier quoted context omitted.

Where is the virtual background option in Jitsi? I couldn't find it.

On Jitsi Meet, in the browser, there is a menu on the bottom right. Click it and there are loads of options there including blur background.

I'm confused. Parent was asking about virtual background. I've checked with meet.jit.si using Chrome and there is no virtual background option in the menu.

Re: Open-Source Virtual Background

#39

Post by TuringNYC »

knotty66 wrote: post #18

Good article, thanks. BTW, this is the method Jitsi-meet uses. They also use BodyPix. https://github.com/jitsi/jitsi-meet/blob/master/react/featur...

Thanks for the share @knotty66. Curious if you can discuss the technicals of this a bit? In the case of Jitsi-meet mobile apps, it seems advantageous that BodyPix models are mobile-first, but did that make sense for the desktop version as well?

I've always been disappointed that BodyPix and some similar models are mobile-first and mobile-only on TensorFlow (https://blog.tensorflow.org/2019/11/updated-bodypix-2.html) -- are these models just not used much on server-side settings? There seems to be very little documentation on doing this server side.

Post reply on HN