Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

github.com

Post reply on HN
Print view View original

111–120 of 245 posts

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#111

Post by kelseyfrog »

refulgentis wrote: post #91
kelseyfrog wrote: post #75

Earlier quoted context omitted.

Ignore the speed comment; it is unrelated to my question. What I mean is, can output be conditioned on antecedent audio as well as text analogous to how image diffusion models can condition inpainting and outpatient on static parts of an image and clip embeddings?

Yes, the paper and Eleven Labs have a major feature of "given $AUDIO_SET, generate speech for $TEXT in the same style of $AUDIO_SET" No, in that, you can't cut it at an arbitrary midword point, say at "what tim" in "what time is it bejing", and give it the string "what time is it in beijing", and have it recover seamlessly. Yes, in that, you can cut it at an arbirtrary phoneme boundary, say 'this, I.S. a; good: test!…

Perfect! Thank you

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#112

Post by stavros »

stevenhuang wrote: post #41

I really want to try this but making the venv to install all the torch dependencies is starting to get old lol. How are other people dealing with this? Is there an easy way to get multiple venvs to share like a common torch venv? I can do this manually but I'm wondering if there's a tool out there that does this.

I generally try to use Docker for this stuff, but yeah, it's the main reason why I pass on these, even though I've been looking for something like this. It's just too hard to figure out the dependencies.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#114

Post by lossolo »

brucethemoose2 wrote: post #102
speedgoose wrote: post #78

Earlier quoted context omitted.

Can it produce any meaningful outputs with such an extreme quantisation?

Yeah, quite good actually, especially if you quantize it on text close to what you are trying to output. Llama 70B is a huge compromise at 2.65bpw... This does make the much "dumber." Yi 34B is much better, as you can quantize it at ~4bpw and still have a huge context.

How would you compare mistral-7b-instruct 16fp (or similar 7b/13b model like llama2 etc) to Yi-34b quantized?

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#115

Post by IshKebab »

refulgentis wrote: post #92
pedrovhb wrote: post #47

Earlier quoted context omitted.

I'm not sure I understand what you mean to say. To me it's a reasonable question asking whether text to speech models can complete a missing part of some existing speech audio, or make it go on for longer, rather than only generating speech from scratch. I don't see a connection to your faster apps analogy. Fwiw, I imagine this is possible, at least to some extent. I was recently playing with xtts and it can generate…

It imposes a cost on others when when you makes false claims like I said or felt the question was unreasonable. I didn't and don't. It is a hard question to understand and an interesting mind-bender to answer. Less policing of the metacontext and more focusing on the discussion at hand will help ensure there's interlocutors around to, at the very least, continue policing.

Sorry but it was pretty obvious what he meant.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#116

Post by fbdab103 »

riquito wrote: post #113
beltsazar wrote: post #62

If AI will render some jobs obsolete, I suppose the first one will be audio book narrators and voice actors.

I can see a future where the label "100% narrated by a human" (and similar in other industries) will be a thing

A la, A Young Lady's Illustrated Primer.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#117

Post by lewismenelaws »

eigenvalue wrote: post #68
eigenvalue wrote: post #63

Earlier quoted context omitted.

Ah ok, thanks. I tried the other demo.

I tried it. Sounds absolutely nothing like my voice or my wife's voice. I used the same sample files as I used 2 days ago on the Eleven Labs website, and they worked flawlessly there. So this is very, very far from being close to "Eleven Labs quality" when it comes to voice cloning.

Yep. Tried as well. Tried a little clip of Tony Sopranos and it came out as a british guy.

xTTSv2 does it much better. But the quality on the trained voices are great though.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#118

Post by eurekin »

lhl wrote: post #95
eurekin wrote: post #57

Earlier quoted context omitted.

Same here. I'm using conda and eyeing simply installing a pytorch into the base conda env

I don't think "base" works like that (while it can be a fallback for some dependencies, afaik, Python packages are isolated/not in path). But even if you could, don't do it. Different packages usually have different pytorch dependencies (often CUDA as well) and it will definitely bite you. The biggest optimization I've found is to use mamba for everything. It's ridiculously faster than conda for package resolution. W…

Thank you! First time I come across. Looks very promising

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#119

Post by brucethemoose2 »

lossolo wrote: post #114
brucethemoose2 wrote: post #102

Earlier quoted context omitted.

Yeah, quite good actually, especially if you quantize it on text close to what you are trying to output. Llama 70B is a huge compromise at 2.65bpw... This does make the much "dumber." Yi 34B is much better, as you can quantize it at ~4bpw and still have a huge context.

How would you compare mistral-7b-instruct 16fp (or similar 7b/13b model like llama2 etc) to Yi-34b quantized?

34B is better. Quantization hurts some, especially in "pro" non chat use cases like RAG, but the increased parameter count makes models so much smarter in comparison.

The perplexity graph here is a pretty good illustration: https://github.com/ggerganov/llama.cpp/pull/1684

YMMV, as Mistral and Yi are not necessarily comparable like different sizes of llama, and it depends on the task.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#120

Post by eigenvalue »

lewismenelaws wrote: post #117
eigenvalue wrote: post #68

Earlier quoted context omitted.

I tried it. Sounds absolutely nothing like my voice or my wife's voice. I used the same sample files as I used 2 days ago on the Eleven Labs website, and they worked flawlessly there. So this is very, very far from being close to "Eleven Labs quality" when it comes to voice cloning.

Yep. Tried as well. Tried a little clip of Tony Sopranos and it came out as a british guy. xTTSv2 does it much better. But the quality on the trained voices are great though.

Yes, same for my voice. Made me sound British and didn't capture anything special about my voice that makes it recognizable.
Post reply on HN