Live data from Hacker News

StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

github.com

81–90 of 245 posts

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#81
post #35
post #26

Earlier quoted context omitted.

A Mac with a lot of unified RAM can do it, or a dual 3090/4090 setup gets you 48gb of VRAM.

Does this actually work? I had thought that you can't use SLI to increase your net memory for the modal?

It works. I use ollama these days, with litellm for the api compatibility, and it seems to use both 24GB GPUs on the server.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#83

Earlier quoted context omitted.

Ah ok, thanks. I tried the other demo.

I tried it. Sounds absolutely nothing like my voice or my wife's voice. I used the same sample files as I used 2 days ago on the Eleven Labs website, and they worked flawlessly there. So this is very, very far from being close to "Eleven Labs quality" when it comes to voice cloning.

The speech generated is the best I've heard from an open source model. The one test I made didn't make an exact clone either but this is still early days. There's likely something not quite right. The cloned voice does speak without any artifacts or other weirdness that most TTS systems suffer from.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#85

If AI will render some jobs obsolete, I suppose the first one will be audio book narrators and voice actors.

Hardly. Imagine licensing your voice to Amazon so that any customer could stream any book narrated in your likeness without you having to commit the time to record. You could still work as a custom voice artist, all with a "no clone" clause if you chose. You could profit from your performance and craft in a fraction of the time, focusing as your own agent on the management of your assets. Or, you could just keep and commit to your day job.

Just imagine hearing the final novel of ASoIaF narrated by Roy Dotrice and knowing that a royalty went to his family and estate, or if David Attenborough willed the digital likeness of his voice and its performance to the BBC for use in nature documentaries after his death.

The advent of recorded audio didn't put artists out of business, it expanded the industries that relied on them by allowing more of them to work. Film and tape didn't put artists out of business, it expanded the industries that relied on them by allowing more of them to work. Audio digitization and the internet didn't put artists out of business; it expanded the industries that relied on them by allowing more of them to work.

And TTS won't put artists out of business, but it will create yet another new market with another niche that people will have to figure out how to monetize, even though 98% of the revenues will still somehow end up with the distributors.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#86
post #82

The quality is really really INSANE and pretty much unimaginable in early 2000s. Could have interesting prospects for games where you have LLM assuming a character and such TTS giving those NPCs voice.

This is a big thing for one area I'm interested in - golf simulation.

Currently playing in a golf simulator has a bit of a post-apocalyptian vibe. The birds are cheeping, the grass is rustling, the game play is realistic, but there's not a human to be seen. Just so different from the smacktalking of a real round, or the crowd noise at a big game.

It's begging for some LLM-fuelled banter to be added.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#88

If AI will render some jobs obsolete, I suppose the first one will be audio book narrators and voice actors.

Hardly. Imagine licensing your voice to Amazon so that any customer could stream any book narrated in your likeness without you having to commit the time to record. You could still work as a custom voice artist, all with a "no clone" clause if you chose. You could profit from your performance and craft in a fraction of the time, focusing as your own agent on the management of your assets. Or, you could just keep and…

What you're not considering here is that a large majority of this industry is made up of no-name voice actors who have a pleasant (but perfectly substitutible) voice which is now something that AI can do perfectly and at a fraction of the price.

Sure, celebrities and other well-known figures will have more to gain here as they can license out their voice; but the majority of voice actors won't be able to capitalize on this. So this is actually even more perverse because it again creates a system where all assets will accumulate at the top and there won't be any distributions for everyone else.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#89

If AI will render some jobs obsolete, I suppose the first one will be audio book narrators and voice actors.

Hardly. Imagine licensing your voice to Amazon so that any customer could stream any book narrated in your likeness without you having to commit the time to record. You could still work as a custom voice artist, all with a "no clone" clause if you chose. You could profit from your performance and craft in a fraction of the time, focusing as your own agent on the management of your assets. Or, you could just keep and…

The point is no one will pay for any of that if you can just clone someone's voice locally. Or just tell the AI how you want it to sound. Your argument literally ignores the entire elephant in the room.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#90
post #29

Earlier quoted context omitted.

It's not called a GAN TTS right? StyleGAN is called what it is because of a "style-based" approach and StyleTTS/2 seems to be doing the same (applying style transfer) through different method (and disentangling style from the rest of the voice synthesis). (Actually, looked at the original StyleTTS paper and it actually even partially uses AdaIN in the decoder, which is the same way that StyleGAN injected style inform…

Yeah no I get this but the naming convention has become so prolific that anyone working in generative space hears "Style " and you should think "GAN". (I work in generative vision btw) My point is not that it is technically right, it is that the name is strongly related with the concept now. Such that if you use a style based network and don't name it StyleX that it's odd and might look like you're trying to claim yo…

So what I'm hearing is... no one should use "style" in its name anymore to describe style transfers because it's too closely associated with a set of models in a sub-field that uses a different concept to apply style that used "style" in its name, unless it also uses that unrelated concept in its implementation? Is that the gist of it, because that sounds a bit mental.

(I'm half kidding, I get what you mean, but also, think about it. The alternative is worse.)

Post reply on HN