Anyone knows at a deeply practical and technical level how 11Labs achieves the level they do?
OpenVoice: Versatile Instant Voice Cloning
191–200 of 207 posts
Re: OpenVoice: Versatile Instant Voice Cloning
#192Earlier quoted context omitted.
> I'll try to ping you Thanks. Use the email address in my profile if anything eventuates. > is an Android app with a keyboard and "speak" button that does API calls to eleven labs sufficient for something worth trying? Maybe. Obviously something with local processing would be preferred, but it might be an option when internet connectivity is good. Is there such an app?
There isn't an ElevenLabs app like that, but I think that's the most expedient method, by far. (i.e. O(days) instead of O(months)) (warning: detailed opinionated take, I suggest skimming) Why? Local inference is hard. You need two things: the clips to voice model (which we have here, but bleeding edge), and text + voice -> speech model. Text to voice to speech, locally, has excellent prior art for me, in the form of…
Wouldn't it be better to use FFI to build an idiomatic interface to use in Dart instead?
Re: OpenVoice: Versatile Instant Voice Cloning
#193My first and ongoing thought is that immoral/criminal uses of voice cloning vastly exceed any legitimate ones.
Talk with your loved ones and make a paraphrase for if you're stuck in a emergency and need money wired or something.
Some banks have voice authentication when you call in and you have to ask to opt out.
Re: OpenVoice: Versatile Instant Voice Cloning
#194Earlier quoted context omitted.
> You don't know what voice cloning is and what it can be used for? * Real-time translation of a user's voice, maintaining emotion and intonation * Professional-quality audio from cheap microphone setups (for video tutorials, indie games, etc.) * Allowing those with speech impairments to communicate using their natural voice again, or: * Allowing those uncomfortable with their natural voice to communicate closer to h…
Most of that can be (and a lot of it was) resolved with artificial voices. There's no need for voice cloning, and certainly no need to make it widely accessible.
> Most of that can be (and a lot of it was) resolved with artificial voices
Existing artificial voices don't even attempt to tackle many of the points and are very clearly a poor substitute (watching a movie dubbed by Alexa?) even in the scenarios where they could be used - far from "resolved". If taking a binary view that it's already technically possible and therefore isn't relevant, I could claim the same of scams.
Given you initially claimed "no positive" but here only take issue to "most" applications listed, do you admit that there are some positives - even if you believe them to be outweighed by negatives?
> certainly no need to make it widely accessible
If it exists but isn't widely accessible, it's likely in the hands of Musk/Zuck and or various state actors. To me that seems the worst alternative - to have a public generally unaware that it's possible and receiving few of the benefits listed, yet still having it available as a tool for competent disinformation.
Re: OpenVoice: Versatile Instant Voice Cloning
#195Earlier quoted context omitted.
There isn't an ElevenLabs app like that, but I think that's the most expedient method, by far. (i.e. O(days) instead of O(months)) (warning: detailed opinionated take, I suggest skimming) Why? Local inference is hard. You need two things: the clips to voice model (which we have here, but bleeding edge), and text + voice -> speech model. Text to voice to speech, locally, has excellent prior art for me, in the form of…
Wouldn't it be better to use FFI to build an idiomatic interface to use in Dart instead?
It's a good point but I'm a perfectionist and can't abide without a web version.
though, now that I write that...
Native: FFI.
Web: Dart calling simple JS function, and the JS handles WASM.
...is an excellent sweet spot. Matches exactly what I do with FONNX. The trouble with WASM is Dart-bounded.
(n.b. re: local cloning for anyone this deep, this would allow local inference of the existing voices in the Raspberry Pi x ONNX voicer project above. It won't _necessarily_ help with doing voice cloning locally, you'll need to prove out that you can get a voice cloning model in ONNX to confirm.)
(n.b. re: translating to Dart, I think the only advantage of a pure Dart port would be memory safety stuff but I also don't think a pure Dart port is feasible without O(months) of time. The C is...very very very 2000s C. globals in one file representing current state that 3 other files need to access. array of structs formed by just reading bytes from a file at runtime that matches the struct layout)
Re: OpenVoice: Versatile Instant Voice Cloning
#196Re: OpenVoice: Versatile Instant Voice Cloning
#197GitHub link: https://github.com/myshell-ai/OpenVoice
Re: OpenVoice: Versatile Instant Voice Cloning
#198Earlier quoted context omitted.
> It's obviously computer generated to my ear. From the README Disclaimer This is an open-source implementation that approximates the performance of the internal voice clone technology of myshell.ai. The online version in myshell.ai has better 1) audio quality, 2) voice cloning similarity, 3) speech naturalness and 4) computational efficiency.
Not totally unexpected unfortunately. Any other OSS players on the market?
RVC
Re: OpenVoice: Versatile Instant Voice Cloning
#199Now of only youtube would ban the use of this crap. Or at the very least allow you filter those videos.
There's genuine uses, look at Apple offering this tech recently as an accessibility feature for people losing the ability to speak to have text to speech in their own voice in lieu of being able to vocalize it themselves. You're banning genuine uses like that or just creators who want to fix a fumbled or awkward line without completely re-recording if you ban it.
He's the same camp that bitches about SD, MJ and DALLe, but loves photoshop generative fill lol
Re: OpenVoice: Versatile Instant Voice Cloning
#200Earlier quoted context omitted.
"Don’t you see that the whole aim of Newspeak is to narrow the range of thought? In the end we shall make thoughtcrime literally impossible, because there will be no words in which to express it."
Censorship by itself doesn't stop people thinking (or even expressing) forbidden thoughts, it stops a person's words reaching other people. BNW had a similar effect by conditioning, rather than by applying the strong form of the Sapir–Whorf hypothesis.
It's not perfectly isometric, but neither is it a stretch to call it orwellian.