Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Infinity – Realistic AI characters that can speak

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 320 posts

Re: Show HN: Infinity – Realistic AI characters that can speak

#41

Post by sidneyprimas »

ianbicking wrote: post #25

The actor list you have is so... cringe. I don't know what it is about AI startups that they seem to be pulled towards this kind of low brow overly online set of personalities. I get the benefit of using celebrities because it's possible to tell if you actually hit the mark, whereas if you pick some random person you can't know if it's correct or even stable. But jeez... Andrew Tate in the first row? And it doesn't g…

Thanks for the feedback! The good news is that the new V2 model will allow people to create their own actors very easily, and so we won't be restricted to the list. You can try that model out here: https://studio.infinity.ai/

The rest of our website still uses the V1 model. For the V1 model, we had to explicitly onboard actors (by fine-tuning our model for each new actor). So, the V1 actor list was just made based on what users were asking for. If enough users asked for an actor, then we would fine-tune a model for that actor.

And yes, the 7s limit on v1 is also a problem. V2 right now allows for 30s, and will soon allow for over a minute.

Once V2 is done training, we will get it fully integrated into the website. This is a pre-release.

Re: Show HN: Infinity – Realistic AI characters that can speak

#44

Post by cchance »

naveensky wrote: post #16

Is it similar to https://loopyavatar.github.io/ . I was reading about this today and even the videos are exactly the same. I am curious if you are anyway related to this team?

Holy shit loopy is good, i imagine another closed model, opensource never gets good shit like that :(

Re: Show HN: Infinity – Realistic AI characters that can speak

#45

Post by lcolucci »

arcticfox wrote: post #38
lcolucci wrote: post #24

Earlier quoted context omitted.

No, not related. We just took some of Loopy's demo images + audios since they came out 2 days ago and people were aware of them. We want to do an explicit side-by-side at some point, but in the meantime people can make their own comparisons, i.e. compare how the two models perform on the same inputs. Loopy is a Unet-based diffusion model, ours is a diffusion transformer. This is our own custom foundation model we've…

This took me a minute - your output demos are your own, but you included some of their inputs, to make for an easy comparison? Definitely thought you copied their outputs at first and was baffled.

Exactly. Most talking avatar papers re-use each others images + audios in their demo clips. It's just a thing everyone does... we never thought that people would think it means we didn't train our own model!

For whoever wants to, folks can re-make all the videos themselves with our model by extracting the 1st frame and audio.

Re: Show HN: Infinity – Realistic AI characters that can speak

#46

Post by andrew-w »

naveensky wrote: post #35

For such models, is it possible to fine-tune models with multiple images of the main actor? Sorry, if this question sounds dumb, but I am comparing it with regular image models, where the more images you have, the better output images you generate for the model.

It is possible to fine-tune the model with videos of a specific actor, but not images. You need videos to train the model.

We actually did this in early overfitting experiments (to confirm our code worked!), and it worked surprisingly well. This is exciting to us, because it means we can have actor-specific models that learn the idiosyncratic gestures of particular person.

Re: Show HN: Infinity – Realistic AI characters that can speak

#47

Post by lcolucci »

zaptrem wrote: post #27

The e2e diffusion transformer approach is super cool because it can do crazy emotions which make for great memes (like Joe Biden at Live Aid! https://youtu.be/Duw1COv9NGQ ) Edit: Duke Nukem flubs his line: https://youtu.be/mcLrA6bGOjY

Nice :) It's been really cool so see the model get more and more expressive over time

Re: Show HN: Infinity – Realistic AI characters that can speak

#48

Post by sidneyprimas »

arcticfox wrote: post #38
lcolucci wrote: post #24

Earlier quoted context omitted.

No, not related. We just took some of Loopy's demo images + audios since they came out 2 days ago and people were aware of them. We want to do an explicit side-by-side at some point, but in the meantime people can make their own comparisons, i.e. compare how the two models perform on the same inputs. Loopy is a Unet-based diffusion model, ours is a diffusion transformer. This is our own custom foundation model we've…

This took me a minute - your output demos are your own, but you included some of their inputs, to make for an easy comparison? Definitely thought you copied their outputs at first and was baffled.

Yes, exactly! We just wanted to make it easy to compare. We also used some inputs from other famous research papers for comparison (EMO and VASA). But all videos we show on our website/blog are our own. We don't host videos from any other model on our website.

Also, Loopy is not available yet (they just published the research paper). But you can try our model today, and see if it lives up to the examples : )

Re: Show HN: Infinity – Realistic AI characters that can speak

#49

Post by andrew-w »

dorianmariefr wrote: post #34

quite slow btw

Yeah, it's about 5x slower than realtime with the current configuration. The good news is that diffusion models and transformers are constantly benefitting from new acceleration techniques. This was a big reason we wanted to take a bet on those architectures.

Edit: If we generate videos at a lower resolution and with a fewer number of diffusion steps compared to what's used in the public configuration, we are able to generate videos at 20-23 fps, which is just about real-time. Here is an example: https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/fast...

Post reply on HN