Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pocket TTS: A high quality TTS that gives your CPU a voice

kyutai.org

Post reply on HN
Print view View original

51–60 of 164 posts

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#51

Post by kube-system »

ethin wrote: post #37
kamranjon wrote: post #20

Earlier quoted context omitted.

That's a pretty crazy requirement for something to be "useful" especially something that runs so efficiently on cpu. Many content creators from non-english speaking countries can benefit from this type of release by translating transcripts of their content to english and then running it through a model like this to dub their videos in a language that can reach many more people.

Uh, no? This is not at all an absurd requirement? Screen readers literally do this all the time, with voices that are the classic way of making a speech synthesizer, no AI required. ESpeak is an example, or MS OneCore. The NVDA screen reader has an option for automatic language switching as does pretty much every other modern screen reader in existence. And absolutely none of these use AI models to do that switching,…

They didn’t say it was a crazy requirement. They said it was crazy to consider it useless without meeting that requirement.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#53

Post by mips_avatar »

jandrese wrote: post #25
singpolyma3 wrote: post #15

Love this. It says MIT license but then readme has a separate section on prohibited use that maybe adds restrictions to make it nonfree? Not sure the legal implications here.

The "prohibited uses" section seems to be basically "not to be used for crime", which probably doesn't have much legal weight one way or another.

I think the only restriction that seems problematic is not being able to clone someone’s voice without permission. I think there’s probably a valid case for using it for satire.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#54

Post by HenrikB »

ethin wrote: post #35
Buttons840 wrote: post #23

Earlier quoted context omitted.

Good question. If a license says "you may use this, you are prohibited from using this", and I use it, did I break the license?

If memory serves, the license is the ultimate source of truth on what is allowed or not. You cannot add some section that isn't in the text of the license (at least in the US and other countries that use similar legal systems) on some website and expect it to hold up in court because the license doesn't include that text. I know of a few other bigger-name projects that try to pull these kinds of stunts because they d…

The copyright holder can set whatever license they want, including writing their own.

In this case, I'd interpret it as they made up a new licence based on MIT, but their addendum makes it non-MIT, but something else. I agree with what others said; this "new" license has internal conflicts.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#55

Post by miki123211 »

jhatemyjob wrote: post #44
echelon wrote: post #24

Earlier quoted context omitted.

What are the advantages of PocketTTS over Kokoro? It seems like Kokoro is the smaller model, also runs on CPU in real time, and is more open and fine tunable. More scripts and extensions, etc., whereas this is new and doesn't have any fine tuning code yet. I couldn't tell an audio quality difference.

Less licensing headache, it seems. Kokoro says its Apache licensed. But it has eSpeak-NG as a dependency, which is GPL, which brings into question whether or not Kokoro is actually GPL. PocketTTS doesn't have eSpeak-NG as a dependency so you don't need to worry about all that BS. Btw, I would love to hear from someone (who knows what they're talking about) to clear this up for me. Dealing with potential GPL contamina…

Kokoro only uses Espeak for text-to-phoneme (AKA G2P) conversion.

If you could find another compatible converter, you could probably replace eSpeak with it. The data could be a bit OOD, so you may need to fiddle with it, but it should work.

Because the GPL is outdated and doesn't really consider modern gen AI, what you could also do is to generate a bunch of text-to-phoneme pairs with Espeak and train your own transformer on them,. This would free you from the GPL license completely, and the task is easy enough that even a very small model should be able to do it.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#56

Post by hexaga »

echelon wrote: post #24
gropo wrote: post #17

Earlier quoted context omitted.

Kokoro is better for tts by far For voice cloning, pocket tts is walled so I can't tell

What are the advantages of PocketTTS over Kokoro? It seems like Kokoro is the smaller model, also runs on CPU in real time, and is more open and fine tunable. More scripts and extensions, etc., whereas this is new and doesn't have any fine tuning code yet. I couldn't tell an audio quality difference.

Kokoro is fine tunable? Speaking as someone who went down the rabbit hole... it's really not. There's no (as of last time I checked) training code available so you need to reverse engineer everything. Beyond that the model is not good at doing voices outside the existing voicepacks: simply put, it isn't a foundation model trained on internet scale data. It is made from a relatively small set of focused, synthetic voice data. So, a very narrow distribution to work with. Going OOD immediately tanks perceptual quality.

There's a bunch of inference stuff though, which is cool I guess. And it really is a quite nice little model in its niche. But let's not pretend there aren't huge tradeoffs in the design: synthetic data, phonemization, lack of train code, sharp boundary effects, etc.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#57

Post by [deleted] »

OfflineSergio wrote: post #48

This is amazing. The audio feels very natural and it's fairly good at handling complext text to speech tasks. I've been working on WithAudio ( https://with.audio ). Currently it only uses Kokoros. I need to test this a bit more but I might actually add it to the app. It's too good to be ignored.

[deleted]

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#58

Post by nowittyusername »

GaggiX wrote: post #2

I love that everyone is making their own TTS model as they are not as expensive as many other models to train. Also there are plenty of different architecture. Another recent example: https://github.com/supertone-inc/supertonic

Thanks for heads up, this looks really interesting and claimed speed is nuts..

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#59

Post by ethin »

kube-system wrote: post #51
ethin wrote: post #37

Earlier quoted context omitted.

Uh, no? This is not at all an absurd requirement? Screen readers literally do this all the time, with voices that are the classic way of making a speech synthesizer, no AI required. ESpeak is an example, or MS OneCore. The NVDA screen reader has an option for automatic language switching as does pretty much every other modern screen reader in existence. And absolutely none of these use AI models to do that switching,…

They didn’t say it was a crazy requirement. They said it was crazy to consider it useless without meeting that requirement.

That doesn't really change what I said though. It isn't crazy to call it useless without some form of ALS either. Given that old school synthesis has been able to do it for like 20 years or so.
Post reply on HN