One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
nvidia-research-mingyuliu.com
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
1–10 of 52 posts
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#2Surprisingly flexible algorithm. I threw a headshot illustration of a comic villain into it and it was able to successfully do all the rotations and even manipulate the eyes despite it not being photorealistic.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#3Wow, this works pretty well.
Makes me think of that chapter in Infinite Jest where videoconferencing gets popular, until people start using "optimized" computer-rendered images instead of showing their actual faces, at which point everyone goes back to audio-only.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#4I'm getting a "Failed to load resource: the server responded with a status of 500 (INTERNAL SERVER ERROR)". Looks like we broke it?
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#5Wow, this works pretty well. Makes me think of that chapter in Infinite Jest where videoconferencing gets popular, until people start using "optimized" computer-rendered images instead of showing their actual faces, at which point everyone goes back to audio-only.
I wouldn't mind video conferencing with computer generated avatars. I don't video conference to know what the other person looks like, and in fact knowing what they look like just creates lots of unnecessary bias. I do it for the cues from their gestures, facial expressions, the direction they are looking, etc. With a good tracking setup that works perfectly well today with digital avatars.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#6I'm getting connection refused errors on the request to
http://54.186.34.220:443/face_vid2vid_rotate
port 443 on http? Hm.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#7This makes a request to server to get the result back. Hacker News hug of death has already happened.
I wish this was deployable to browsers so it was fully stand alone.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#8The side by side against other algos is pretty cool:
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#9Surprisingly flexible algorithm. I threw a headshot illustration of a comic villain into it and it was able to successfully do all the rotations and even manipulate the eyes despite it not being photorealistic.
There was an app going around a few months ago called “wombo.ai” that will make a headshot sing. It had the same outcome when given non human pictures as a source. Easily killed an afternoon trying out different things.
Re: One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
#10While the client-side has input validation, it appears that the server-side does not, as I can edit the request body freely and it'll return accordingly.
It's interesting to see how the model fails at extreme values. I can see why they chose the cutoffs they did!