Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I turned my face rec system into a video codec

vertigo.ai

Post reply on HN
Print view View original

91–100 of 150 posts

Re: Show HN: I turned my face rec system into a video codec

#91

Post by akrymski »

Well done for putting it out there!

We've worked on this about 3 years ago, plus background removal (realtime alpha-matting still not really done well by anyone), portrait re-lighting (G-Meet is now doing this) and even eye-contact (adjusting eye position to create the illusion of looking directly into the camera).

Some findings:

- Competing with dedicated H264/5 chips is very hard, especially when it comes to energy efficiency, which is something that mobile users ultimately care a lot about. Super-resolution on H264 is probably the way to go.

- It's hard to turn this into a business (corporates that pay for Zoom don't seem to care much).

PS Also a big fan of super-tiny AI models (binarized NNs, frequency-domain NNs, etc) for edge applications. Happy to chat!

Re: Show HN: I turned my face rec system into a video codec

#92

Post by nicr_22 »

Nice work, but you might find it's not super unique - video codec people have been thinking about how to apply face recognition ML tech to this use case for 5+ years.

For instance, have you seen https://developer.nvidia.com/maxine ? They released some pretty nice demos 2 years ago.

Re: Show HN: I turned my face rec system into a video codec

#94

Post by telesilla »

willy_k wrote: post #90
bambax wrote: post #48

> Because it uses face rec, you can ONLY show your face, and if you disappear from view your audio will stop after a while, to avoid situations like when you need to go the the restroom but forget to mute. Of course, the real killer app for Zoom calls is the opposite of this: some kind of deep fakery that makes it seem we're there when we're not. Yet as it is, this is a fantastic idea. It's surprising video codecs de…

Well if you just want to trick people into thinking you’re present when you aren’t, you can just use a video file as your camera feed. There are a bunch of tools to do this. Now, the main issue would be if you want to speak using this setup, but I guess one solution to that would be to use a model to lip sync your non-speaking video on-the-fly, which seems to be being discussed here[0]. [0] https://github.com/Rudrabh…

OBS virtual camera is another easy way. Load a video of you paying attention and have a quick key assigned to a scene of your live cam if needed.

Re: Show HN: I turned my face rec system into a video codec

#95

Post by pvillano »

bambax wrote: post #48

> Because it uses face rec, you can ONLY show your face, and if you disappear from view your audio will stop after a while, to avoid situations like when you need to go the the restroom but forget to mute. Of course, the real killer app for Zoom calls is the opposite of this: some kind of deep fakery that makes it seem we're there when we're not. Yet as it is, this is a fantastic idea. It's surprising video codecs de…

OBS allows you to create a virtual webcam, where you can use the live compositor to switch between, say, a looping video of you sitting still and your actual webcam. Smoke and mirrors could include a low frame rate and heavy compression to hide the transition

Re: Show HN: I turned my face rec system into a video codec

#96

Post by jacobgorm »

kordlessagain wrote: post #70
cupofpython wrote: post #69

Earlier quoted context omitted.

>and if you disappear from view your audio will stop after a while I sure hope this "feature" can be turned off in the settings

Wouldn't this be an "auto mute"? I haven't tried it, but maybe it unmutes when it sees your face again.

Yeah that is a feature, but I have plans for adding a button that disables it. I like not having to worry about muting my mic when I leave the camera, but there are times where you want the opposite.

Re: Show HN: I turned my face rec system into a video codec

#97

Post by jacobgorm »

nicr_22 wrote: post #92

Nice work, but you might find it's not super unique - video codec people have been thinking about how to apply face recognition ML tech to this use case for 5+ years. For instance, have you seen https://developer.nvidia.com/maxine ? They released some pretty nice demos 2 years ago.

Their approach is more heavy-weight as it uses GANs (IIRC) to dream up a reconstruction of your face. They need GPU VMs in the cloud, whereas mine runs on device.

Re: Show HN: I turned my face rec system into a video codec

#98

Post by jacobgorm »

bsnal wrote: post #93

Do you have any benchmarks? Because this looks like it will be unusable on the average $300 windows laptop.

I currently have (the prototype port of) it running on my T14-2 (11th gen i5) and my 2018 Mac Mini without problems, and have it confirmed to work back to 2015 macs, but older laptops may suffer. The main problem the cheap laptops (and even my quite expensive T14) have is that the cameras are absolutely horrible compared to anything Apple would ship.

Re: Show HN: I turned my face rec system into a video codec

#99

Post by kleer001 »

I love the idea and I hope it succeeds.

Only one small bit of cosmetic feedback. Maybe think about hiring a face model to work your demo. I personally don't care, but I think it might improve your optics. And yes, I understand it's a tech demo. But booth babes were a thing (are they still a thing?) for good reason, grab those eyeballs, yo.

Post reply on HN