Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PlayAI's new Dialog model achieves 3:1 preference in human evals

play.ht

Post reply on HN
Print view View original

1–10 of 59 posts

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#3

Post by legofan94 »

vessenes wrote: post #2

So, this is really impressive. Expressivity and pacing are wayyy better. Eleven Labs has been tops for some time, but the difference is pretty remarkable!

Thanks Peter! We think it really crushes for emotive text. Anything from storytelling to being emotionally reassuring. Still a lot of things up our sleeve too!

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#4

Post by vessenes »

legofan94 wrote: post #3
vessenes wrote: post #2

So, this is really impressive. Expressivity and pacing are wayyy better. Eleven Labs has been tops for some time, but the difference is pretty remarkable!

Thanks Peter! We think it really crushes for emotive text. Anything from storytelling to being emotionally reassuring. Still a lot of things up our sleeve too!

I have a particular use case I’m interested in using agents for - any chance you want to have a call?

In brief I’d like to be able to generate conversations via api choosing voices that should be unique on the order of thousands. Essentially I’m trying to simulate conversations in a small town. Eleven is not set up for this.

Ideally I’d be able to pick a spot in latent space for a voice programmatically. But I’m open to suggestions.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#9

Post by thot_experiment »

I've been messing with the open source side of audio generation, and expressiveness still takes work but it's getting there. Roughly summarized my findings are:

- zero shot voice cloning isn't there yet

- gpt-sovits is the best at non-word vocalizations, but the overall quality is bad when just using zero shot, finetuning helps

- F5 and fish-speech are both good as well

- xtts for me has had the best stability (i can rely on it not to hallucinate too much, the others i have to cherrypick more to get good outputs)

- finetuning an xtts model for a few epochs on a particular speaker does wonders, if you have a good utterance library w/ emotions conditioning a finetuned xtts model with that speaker expressing a particular emotion yields something very usable

- you can do speech to speech on the final output of xtts to get to something that (anecdotally) fools most of the people i've tried it on

- non finetuned XTTS zero shot -> seed-vc generates something that's okay also, especially if your conditioning audio is really solid

- really creepy indistinguishable at a casual listen voiceclones of arbitrary people are possible with as little as 30 minutes of speech, the resultant quality captures mannerisms and pacing eerily well, it's easy to get clean input data from youtube videos/podcasts using de-noising/vocal extraction neural nets

TL;DR; use XTTS and pipe it into seed-vc, the e2e on that pipeline on my machine is something like 2x realtime and generates very highly controllable natural sounding voices, you have to manually condition emotive speech

Post reply on HN