11–20 of 150 posts
There’s some weird banding/rainbow effects around my glasses and the background (not on my face), but that’s the only major artifact that stood out to me.
These kinds of failure modes for "AI" are the most interesting to me.
It seems extremely smart - but don’t you think that to have success in a mass market product - think like MS Teams - it would need to be a combined solution. Where it both can do this for faces, efficiently, but also continues to work in a predictable way if I want to show an item/page from a book/my cat/my kid to other people in the call?
This is just a phenomenal idea - I hope your patent is approved too!
>> This also solves dick-pics etc.
Is this a problem on zoom meetings, for people other than Jeffrey Toobin?
Firstly, this is pretty awesome, love it. I have a few questions: * I applaud the work to have it run on tiny-bandwidths, how hard would it be to up the frame-rate to 60? * How well does "framing" work? Are you able to add flexible amounts of padding around the head or is very focussed on a face taking up the whole canvas? * How much does it "cheat". Is it firing only feature-maps so if I have a spot on my chin does…
Wrt upping the frame rate the main problem is that the phone may run a bit hot, newer iPhones/iPads should be able to handle it just fine, but the older ones based on, say, the A10, might have trouble keeping up, especially with multiple remote parties connected.
* The framing depends on a transformation derived from the face landmarks, and the amount of padding is somewhat flexible. Distance from the camera seems to impact this, so it could be that my landmarks model needs some tweaking to work better when you are sitting very close to the camera.
* This is closer to being a general video codec than a face-generating GAN, so there is not a lot of "cheating" in that respect. It is optimized for transmission of faces, but other images will pass through if you let them (which I currently don't).
* I built the AI engine and the face recognizer etc from scratch, though with the help of a former co-founder who was originally the one training our models (in pytorch). The vertigo.ai home page has some demo videos. We initially targeted raspberry-pi style devices, NVIDIA Jetsons, etc., but have since ported to IOS and MacOS. Our initial customers were startups, mostly in the US, and a large Danish university that uses us for auditorium head counting.
* It empirically does seem to work on diverse faces, both in real life and when testing on for example the "coded bias" trailer. Ideally I would like to test more systematically on something like Facebook/Meta's "casual conversations" dataset.
What is the invention? The models are just complex mathematical formulas and these cannot be patented.
I bit my tongue off at one point in my life (jumped from a high height and my knee hit my chin as I was screaming). The fact that it captures most of the details of the scar where it was reattached is phenomenal. Majorly impressed. There’s some weird banding/rainbow effects around my glasses and the background (not on my face), but that’s the only major artifact that stood out to me.
Glasses are sometimes a little bit of a problem, I don't have enough of those in my training sets.
But the story has a deeper meaning for me (because of the book I am writing). You switched from street face surveillance (an arguably highly unethical application) to more intimate videoconferencing (a more humanistic and socially beneficial end).
May I ask you in all sincerity, what if any ethical considerations played a part in your change of direction?
I suspected from the title to read at least some sub-text that you turned your back on mass-surveillance to find a "better" use for your work. But you express no value judgements and only really mention that the pandemic took away potential targets.
> is a new from-the-ground-up patent pending What is the invention? The models are just complex mathematical formulas and these cannot be patented.
I am not a super fan of patents, but for background please consider that Asger Jensen and I could have patented VM live migration in 2002 and chose not to for idealistic reasons, just to see VMware do it.
Does it have any interesting "nonlinear" effects where it can show an entirely different face (the wrong face) based on misidentification or even adversary input? These kinds of failure modes for "AI" are the most interesting to me. It seems extremely smart - but don’t you think that to have success in a mass market product - think like MS Teams - it would need to be a combined solution. Where it both can do this for…