Live data from Hacker News

OpenVoice: Versatile Instant Voice Cloning

arxiv.org

171–180 of 207 posts

Re: OpenVoice: Versatile Instant Voice Cloning

#171

Earlier quoted context omitted.

Well we could just look at the obvious and existing use cases for text to speech stuff. Alexa, siri, and similar, are all common place. Another huge usecase would be anything to do with voice acting. Either in video games, cartoons, or the like. This would completely democratize voice acting material, and would empower anyone to be able to do this for cheap.

... and put 99% of voice actors out of business. We'll eventually end up with every TV show, movie, and, video game being voiced by Ryan Gosling and Beyonce because market research.

Technology that makes things better, faster, and more democratized does tend to harm those who profit off of things being expensive and gatekept, yes.

Democratization will always be the enemy of those who profit from preventing others from being empowered.

Re: OpenVoice: Versatile Instant Voice Cloning

#172
post #149

Earlier quoted context omitted.

> It's obviously computer generated to my ear. From the README Disclaimer This is an open-source implementation that approximates the performance of the internal voice clone technology of myshell.ai. The online version in myshell.ai has better 1) audio quality, 2) voice cloning similarity, 3) speech naturalness and 4) computational efficiency.

So this paper is a thinly veiled ad of myshell.ai's services?

Like 50% of arxiv. SV figured out that people read papers in 202x, not PRNewsWire and have adjusted accordingly.

Re: OpenVoice: Versatile Instant Voice Cloning

#173

It's not really well advertised and I'm not sure Apple is continuing development, but iOS has a voice clone feature called "Personal Voice" - it takes about 15 mins to train it with your own voice (and then takes a few hours to process on-device when locked). You can use it in phone calls and FaceTime (maybe other places?). It would be nice to use it for general TTS.

It's an accessibility feature for people losing their voice or on the verge of. And it is TTS only, not speech-to-speech as your mention of "can use it in phone calls and facetime" implies. Not being s2s means it doesn't retain vocal disfluencies, prosody etc signals that make a voice feel real

Re: OpenVoice: Versatile Instant Voice Cloning

#174
post #75

I commend the authors on making this easy to try! However it doesn't work very well for me for general voice cloning. I read the first paragraph of the wikipedia page on books and had it generate the next sentence. It's obviously computer generated to my ear. Audio sample: https://storage.googleapis.com/dalle-party/sample.mp3 Cloned voice (converted to mp3): https://storage.googleapis.com/dalle-party/output_en_defaul…

> It's obviously computer generated to my ear. From the README Disclaimer This is an open-source implementation that approximates the performance of the internal voice clone technology of myshell.ai. The online version in myshell.ai has better 1) audio quality, 2) voice cloning similarity, 3) speech naturalness and 4) computational efficiency.

Not totally unexpected unfortunately. Any other OSS players on the market?

Re: OpenVoice: Versatile Instant Voice Cloning

#175
post #109
post #45

Earlier quoted context omitted.

You don't know what voice cloning is and what it can be used for? These people made it even easier to do. That's their achievement. Facilitating fraud, confusion and distrust.

> You don't know what voice cloning is and what it can be used for? * Real-time translation of a user's voice, maintaining emotion and intonation * Professional-quality audio from cheap microphone setups (for video tutorials, indie games, etc.) * Allowing those with speech impairments to communicate using their natural voice again, or: * Allowing those uncomfortable with their natural voice to communicate closer to h…

Most of that can be (and a lot of it was) resolved with artificial voices. There's no need for voice cloning, and certainly no need to make it widely accessible.

Re: OpenVoice: Versatile Instant Voice Cloning

#176

If I wanted to do voice cloning on my own hardware can anyone suggest what a good open source project would be to use? What is the state of the art in open source voice cloning?

I use Tortoise TTS. It's slow, a little clunky, and sometimes the output gets downright weird. But it's the best quality-oriented TTS I've found that I can run locally. It's allegedly the basis of the tech used by Eleven Labs. https://github.com/neonbjb/tortoise-tts

There are faster implementations of tortoise that allow fine-tuning. You can get close to ElevenLabs quality if you have a perfect dataset. https://git.ecker.tech/mrq/ai-voice-cloning

Re: OpenVoice: Versatile Instant Voice Cloning

#177

Earlier quoted context omitted.

> Why does it need a practical reason? To at least give us something as a consolation for all the havoc all sorts of deep fakes will wreak on societies. It's like asking what a knife can be used for other than murder. It's a valid question.

It's a valid question, but not a good one. The implication is something needs a reason to exist. This is a new technology, and just like all new technologies, the fear of it is spreading faster than the understanding. Scams aren't going away. Will this make it easier to scam some people? Absolutely, so did the internet. I'm not claiming this is anything like the internet. My argument closer to, the reason people get…

> But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live.

> If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used.

-- Joseph Weizenbaum

That is not fear. That is being serious and unflinching, if anything.

> We can all wring our hands in fear over what the new technology might do, or we can solve the problems we care about.

I'm doing neither. I said it's a valid question, with which you agree. The rest is a straw man apropos nothing anyone actually said, here, and wringing your hands about it. It's a way bigger waste of time than asking a simple question and let those who want to answer that, and let those who don't want answer it simply don't answer it, instead of making up this "issue" with the question itself.

> Authenticity was hard before this, and it'll be hard after.

So "nothing changes", but technology is super important? You could say the same about, say, curing cancer. People will live for a while and then die, with or without it. Why since it makes no difference, what'd be the problem with "fearing" it?

Re: OpenVoice: Versatile Instant Voice Cloning

#178
post #98
post #60

Earlier quoted context omitted.

> Trust nothing > This is a society-destroying idea. Believe it or not, this is how much of the population saw The Internet when it first came close to being mainstream. Everyone and their mother said "Don't believe anything you read on the cybernet", which ended up ironic as everyone and their mother ended up being the ones to believe anything on the cybernet anyways. > everyone will live in a warped and fragile alt…

It's vastly different than today because today (or at least a few years ago), I could trust videos and voices delivered digitally. I can't do that anymore.

How long has society had voice and video delivered digitally? We managed to survive fine before we had it.

If it now becomes impossible to trust a voice received through the internet without being connected to a verified telephone number I don't know how that can be classified as society-changing.

Re: OpenVoice: Versatile Instant Voice Cloning

#179

Earlier quoted context omitted.

Your friend can take a look at solutions from Acapela [0], SpeakUnique [1] or VOCALiD [2]. Not sure whether they have a solution for Android though. I recently saw a video from google about a custom voice they created for somebody with ALS but I can't seem to find it online (Does anybody have a link?). Creating custom voices is not yet available on Android though. The latest iOS release (iOS 17) does support creating…

Hi and thanks for the suggestions. Looking through them, it looks like you need to do what they call "voice banking" before you lose your voice. Basically reading a script they provide. Unfortunately my friend's voice is too far gone for that to be possible. Hoping for something where they can use old recordings to generate a voice.

From my tests I think Audiobox from Meta is the most promising (even better than Eleven Labs) - too bad it's closed source and they force you to read some randomly generated sentences (to prevent the case of someone generating a cloned voice without consent).

Right now Eleven Labs is your best bet.

xTTS is just not there quality wise. The version available in the studio is marginally better than the OSS version but it's still pretty far from being believable.

The non-nerfed version of Tortoise (the author decided to ruin their own project but forks exist) was decent at voice cloning but it takes a lot of tries.

I'm pretty sure we already have the technology to do what you want and help your friend, it's just a matter of time until it gets better and more software comes out.

Re: OpenVoice: Versatile Instant Voice Cloning

#180
post #157

Earlier quoted context omitted.

How? Your bank stores personal data covered by gdpr but enabling crappy secure systems is not the domain of gdpr. Most likely this is caused by SCA another European directive that ruined our lives with extra security hoops (for payment providers) for little extra security - or even worse in case of voice password or security questions

Sadly, the US, where I currently live, is quite behind in this. Considering going expat (not for this specific reason, but it doesn't help); any expats have recommendations of what countries have worked well for them?

Heavily depends on money available / crime level tolerance - albeit crime level in the US is pretty crazy.

Europe is deteriorating incredibly rapidly in terms of crime (due to a combination of economic poverty and uncontrolled immigration from third world countries) - but I think some of the low tax EU countries (Malta, Cyprus, Gibraltar, etc) are a good bet for a few more years.

My top choice if I had family (or friends I want to be close with) in the US would be Cayman.

I think long term, either South America drops the level of crime considerably and becomes the new place to be or China start building futuristic cities attracting wealthy western talent to offset their declining population rate.

Post reply on HN