Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

github.com

Post reply on HN
Print view View original

211–220 of 245 posts

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#211

Post by maeil »

slow_numbnut wrote: post #206
kats wrote: post #202

Earlier quoted context omitted.

That's not really true, most scammers have a male voice with a heavy accent. When they have tools that easily disguise their voice, scammers can reach many more elderly people.

That might have been true about a year ago, but I've been getting calls from well-spoken native-level scammers for about two months now. They are so frequent that I can put them on speaker during family gatherings to raise awareness. Sample sizes of 1 are never representative but they definitely have full access to native speakers or tech that can generate very passable speech.

It seems quite possible that the change you've seen in these last two months is because some have started using these models. More likely than a sudden huge shift in either the country of origin or English skills of the scammers.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#212

Post by beltsazar »

washadjeffmad wrote: post #127
nikkwong wrote: post #88

Earlier quoted context omitted.

What you're not considering here is that a large majority of this industry is made up of no-name voice actors who have a pleasant (but perfectly substitutible) voice which is now something that AI can do perfectly and at a fraction of the price. Sure, celebrities and other well-known figures will have more to gain here as they can license out their voice; but the majority of voice actors won't be able to capitalize o…

No, I am. I work with them, and I've been one (am one, rarely). I listed just one possible use, but I also see voice cloning and advanced TTS expanding access for evocative instruction, as an aid to study style and expand range. Don't be afraid on their behalf. The dooming you're talking about applied to every one of the technological changes I already listed, and we employ more performers and artists today than ever…

> When animation went digital, we graduated more storyboard artists and digital animators. When music notation software and sampling could replace musicians and orchestras, we graduated more musicians and composers trained on those tools.

What you explained is that tech has changed the tools used by artists.

It's substantially different with AI-based TTS, though. It's not a tool for artists, but it's a tool for movie/game/book publishers to replace human voice actors. The AI will be much much more scalable and cheaper.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#213

Post by IanCal »

modeless wrote: post #187
woodson wrote: post #183

Earlier quoted context omitted.

A simple correlation of audio chunks from microphone and from the TTS should be enough to tell which parts in the input stream are re-recorded TTS. Much simpler, no?

It's not so simple when the impulse response of the room and mic and speakers are all unknown, possibly changing, plus unknown background sounds as well, possibly at a very high level. and there's also unknown latency which can be quite large especially in the networked case, and maybe some codecs, and maybe some audio "enhancement" software the OEM installed on the user's machine. Also, ideally the computer would be…

Could it be done more reasonably with the transcription? With diarisation the logic of "someone is saying exactly / almost exactly what I'm saying, that's probably me" might be pretty reasonable.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#214

Post by KolmogorovComp »

wczekalski wrote: post #55
eigenvalue wrote: post #50

Was somewhat annoying to get everything to work as the documentation is a bit spotty, but after ~20 minutes it's all working well for me on WSL Ubuntu 22.04. Sound quality is very good, much better than other open source TTS projects I've seen. It's also SUPER fast (at least using a 4090 GPU). Not sure it's quite up to Eleven Labs quality. But to me, what makes Eleven so cool is that they have a large library of high…

One thing I've seen done for style cloning is a high quality fine tuned TTS -> RVC pipeline to "enhance" the output. TTS for intonation + pronunciation, RVC for voice texture. With StyleTTS and this pipeline you should get close to ElevenLabs.

RVC? R… Voice Model?

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#215

Post by Monicjames »

So, we've got this open-source TTS wizardry going on, which is kinda like if Siri had a caffeine overdose - faster, snappier, and way more fun at parties. This thing is running on gaming rigs with beefy GPUs, and it's apparently so user-friendly, even your grandma could set it up without accidentally summoning a digital demon.

But here's the real kicker - it's got the manners of a Victorian gentleman. You can rudely interrupt it mid-sentence, and it'll just stop and listen. Politeness level 100. The reverse, though - getting Mr. Bot to interrupt you - is still in the 'that's too much brain for my silicon' phase. Like, how do you teach a bunch of 1s and 0s to know when you're just taking a dramatic pause or actually done with your TED talk?

And get this - they're talking about making this bot read body language. Imagine your laptop judging you for your slouchy posture or that 'I haven't slept properly in days' look. Creepy? Maybe a bit. Cool? Absolutely.

In conclusion, StyleTTS2 is shaping up to be the cool new kid on the block, but it's still learning the ropes of human conversation. It's like that super smart friend who knows everything about quantum physics but can't tell when you're sarcastically saying 'Yeah, sure, let's invade Mars tomorrow.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#216

Post by Jach »

TOMDM wrote: post #124
tomp wrote: post #122

Earlier quoted context omitted.

How do you get Whisper to be fast? Isn't it quite non-realtime?

The community upgrades to whisper are far faster than real-time, especially if you have a powerful gpu

There's several faster ones out there. I've been using https://github.com/Softcatala/whisper-ctranslate2 which includes a nice --live_transcribe flag. It's not as good as running it on a complete file but it's been helpful to get the gist of foreign language live streams.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#217

Post by _lvbh »

I am an introvert: I rarely socialize, listen to podcasts at 2x speed, and mostly use subtitles rather than listening to audio for movies; therefore having a below average ability to differentiate humans/robots.

I asked someone to play the recordings for me to differentiate. I could not tell which was human (only between StyleTTS2 and Ground truth. The others were obvious)

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#218

Post by _lvbh »

deknos wrote: post #204

Is this really opensource and/or free software? like code, data(set/s) and models? I am quite tired to see some "open-source" advertisement, where the half or more is not really free. general psa: please be honest in your announcements :|

MIT licensed. Models, code, and everything is available right there when you click the link.

Maybe actually check it out before complaining.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#219

Post by _lvbh »

exizt88 wrote: post #123

The weights aren’t MIT-licensed, so this is not usable in commercial applications, right?

It is usable in commercial applications given you disclose the use of AI. This applies only to the pre-trained models. You can train your own from scratch without these restrictions.

You can fine tune it on your own voice and also not be required to disclose the use of AI.

Re: StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

#220

Post by mx20 »

kats wrote: post #199

This is really harmful and unethical work. It will be used to hurt millions of elderly people with scams. That's the real application that will happen 100x more than anything else. It's unethical and harmful to release tools that will be overwhelmingly used to hurt elderly people. What they should do about it is: Stop releasing models. Only release a service so that scammers will not use it. Also, only released audio…

Scammers scamming old people is already very wide spread, so should we maybe outlaw telephones as well? Or maybe mandate anti scamming filters that disconnect if something is discussed that could be a scam? If I think about it that actually would make more sense, but still be problematic.
Post reply on HN