Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-Source Virtual Background

elder.dev

Post reply on HN
Print view View original

81–85 of 85 posts

Re: Open-Source Virtual Background

#81

Post by wasmitnetzen »

rogierhofboer wrote: post #80
BenTheElder wrote: post #30

Earlier quoted context omitted.

I need to profile it more closely, I actually don't remember the exact FPS etc. but I don't expect that to be the limiting factor. The inference / ml is expensive (which I did profile initially...), and I suspect not really optimized on this backend. It appears to be faster with webGL in the browser. I sorta stopped worrying about it once it was "good enough" to show up to a few meetings with, but with all the attent…

I had success by replacing the get_mask function with: from keras.models import load_model model = load_model('models/transpose_seg/deconv_bnoptimized_munet.h5', compile=False) def get_mask(frame): # Preprocess frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) simg = cv2.resize(frame, (128, 128), interpolation=cv2.INTER_AREA) simg = simg.reshape((1, 128, 128, 3)) / 255.0 # Predict out = model.predict(simg) # Postprocess…

Thanks! That helped me in my attempt at this: https://github.com/wasmitnetzen/dynjandi

Re: Open-Source Virtual Background

#82

Post by soylentcola »

bronco21016 wrote: post #70
soylentcola wrote: post #66

Earlier quoted context omitted.

I mean...in fairness I could probably figure out how to get hold of a sheet (seriously, of all the ugly colored sheets and blankets they sell at the local Target, there isn't a bright green or even blue??) Likewise there are loads of cheap ones on Amazon or I could even get some bright paint and do a wall (or maybe dye my current non-chromakey white sheet background). It's just a matter of suspended/delayed deliverie…

Been struggling with this myself the last few days! I don’t NEED to go out or order all of this fancy video conferencing equipment for my weekend virtual happy hour but it’s way more fun being locked inside if I do. I was fortunate enough that the previous owners left behind a god-awful teal paint that works amazingly as a ‘blue’ chromakey. For a huge upgrade in camera I’m using the OBS Camera app on iOS which wirele…

My lighting is currently a mix of a clamp-on utility light with parchment paper diffuser (key), a window (fill), and a cheap old desk gooseneck lamp pointed at my sheet (I had used it as cheapo outdoor movie screen and now it acts as a clean background for current VTC needs).

Cam is the basic 1280x720 Microsoft Lifecam USB webcam I've had for years but is still much better than any laptop cam due to placement flexibility and better optics. OBS lets me color correct and exposes the manual controls for focus, exposure, etc.

I did grab a USB headset from work before work-from-home started, but I found it had a frayed wire and isn't any good. Instead I am using my (wired) cell headphones with their integrated mic instead of the USB headset.

I dug out an old USB audio interface I had packed away and tried it with a Shure vocal mic, but that led me down another hole of messing with Voicemeeter to tweak EQ and noise reduction because it's really meant to be held right up to your mouth (for singing, etc) and picks up background noise if I have the levels up high enough to use as a desk/stand mic. For the time being I am sticking with the headphone/mic for simplicity's sake.

None of this is what I'd use if I had been handed a couple hundred bucks to put together a VTC setup but it's all stuff I had laying around and looks/sounds so much better than everyone who's just using built-in laptop cam/mic/speakers.

Re: Open-Source Virtual Background

#83

Post by dh2424 »

dhimes wrote: post #53
dh2424 wrote: post #49

This is pretty awesome work, but I just wanted to point out that Zoom doesn't actually require a green screen. If you uncheck the "I have a green screen" button, choosing your own virtual background still looks really good, although of course you're not going to get any crazy effects like this script adds.

Depends on your hardware. I have a Win machine, but need a green screen because I don't have the horsepower.

Ah ok, didn't know that.

Re: Open-Source Virtual Background

#84

Post by oliv3r »

BenTheElder wrote: post #24
villgax wrote: post #22

Pretty insane to use Docker & web-requests for high frame rate video chat with something which runs just fine even on CPU.

Docker made it easier to package up the dependencies (especially aligning cuda etc.), and containers are my dayjob :-) The web requests are just an easy mode of IPC to pass around some bags of bytes, "high frame rate" is at most 30 qps ... that part isn't really interesting performance wise and this isn't a production tool :-) I'm not sure I'd be so confident about tensorflow.js being so fast on the CPU ... you can s…

So this is super cool, and I would love to use it, but sadly, in your write-up you mention _requires_ nVidia hardware, which sadly, I don't have :(

Any follow up on making this more generic, e.g. with AMD/Intel setups?

Post reply on HN