Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pocket TTS: A high quality TTS that gives your CPU a voice

kyutai.org

Post reply on HN
Print view View original

21–30 of 164 posts

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#21

Post by satvikpendem »

daemonologist wrote: post #19
syntaxing wrote: post #9

Is there something similar for STT? I’m using whisper distill models and they work ok. Sometimes it gets what I say completely wrong.

Parakeet is not really more accurate than Whisper, but it's much faster - faster than realtime even on CPU: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 . You have to use Nemo though, or mess around with third-party conversions. (Also has a big brother Canary: https://huggingface.co/nvidia/canary-1b-v2 . There's also the confusingly named/positioned Nemotron speech: https://huggingface.co/nvidia/nemotron-speech…

Keep in mind Parakeet is pretty limited in the number of languages it supports compared to Whisper.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#22

Post by Levitz »

phoronixrly wrote: post #12
dust42 wrote: post #11

Good quality but unfortunately it is single language English only.

I echo this. For a TTS system to be in any way useful outside the tiny population of the world that speaks exclusively English, it must be multilingual and dynamically switch between languages pretty much per word. Cool tech demo though!

But it wouldn't be for those who "speak exclusively English", rather, for those who speak English. Not only that but it's also common to have system language set to English, even if one's language is different.

There's about 1.5B English speakers in the planet.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#23

Post by Buttons840 »

singpolyma3 wrote: post #15

Love this. It says MIT license but then readme has a separate section on prohibited use that maybe adds restrictions to make it nonfree? Not sure the legal implications here.

Good question.

If a license says "you may use this, you are prohibited from using this", and I use it, did I break the license?

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#24

Post by echelon »

gropo wrote: post #17
armcat wrote: post #4

Oh this is sweet, thanks for sharing! I've been a huge fan of Kokoro and event setup my own fully-local voice assistant [1]. Will definitely give Pocket TTS a go! [1] https://github.com/acatovic/ova

Kokoro is better for tts by far For voice cloning, pocket tts is walled so I can't tell

What are the advantages of PocketTTS over Kokoro?

It seems like Kokoro is the smaller model, also runs on CPU in real time, and is more open and fine tunable. More scripts and extensions, etc., whereas this is new and doesn't have any fine tuning code yet.

I couldn't tell an audio quality difference.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#25

Post by jandrese »

singpolyma3 wrote: post #15

Love this. It says MIT license but then readme has a separate section on prohibited use that maybe adds restrictions to make it nonfree? Not sure the legal implications here.

The "prohibited uses" section seems to be basically "not to be used for crime", which probably doesn't have much legal weight one way or another.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#26

Post by CGamesPlay »

singpolyma3 wrote: post #15

Love this. It says MIT license but then readme has a separate section on prohibited use that maybe adds restrictions to make it nonfree? Not sure the legal implications here.

For reference, the MIT license contains this text: "Permission is hereby granted... to deal in the Software without restriction, including without limitation the rights to use". So the README containing a "Prohibited Use" section definitely creates a conflicting statement.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#27

Post by echelon »

phoronixrly wrote: post #12
dust42 wrote: post #11

Good quality but unfortunately it is single language English only.

I echo this. For a TTS system to be in any way useful outside the tiny population of the world that speaks exclusively English, it must be multilingual and dynamically switch between languages pretty much per word. Cool tech demo though!

English has more users than all but a few products.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#28

Post by phoronixrly »

Levitz wrote: post #22
phoronixrly wrote: post #12

Earlier quoted context omitted.

I echo this. For a TTS system to be in any way useful outside the tiny population of the world that speaks exclusively English, it must be multilingual and dynamically switch between languages pretty much per word. Cool tech demo though!

But it wouldn't be for those who "speak exclusively English", rather, for those who speak English. Not only that but it's also common to have system language set to English, even if one's language is different. There's about 1.5B English speakers in the planet.

Let's indeed limit the use case to the system language, let's say of a mobile phone.

You pull up a map and start navigation. All the street names are in the local language, and no, transliterating the local names to the English alphabet does not make them understandable when spoken by TTS. And not to mention localised foreign names which then are completely mangled by transliterating them to English.

You pull up a browser, open up an news article in your local language to read during your commute. You now have to reach for a translation model first before passing the data to the English-only TTS software.

You're driving, one of your friends Signals you. Your phone UI is in English, you get a notification (interrupting your Spotify) saying 'Signal message', followed by 5 minutes of gibberish.

But let's say you have a TTS model that supports your local language natively. Well due to the fact that '1.5B English speakers' apparently exist in the planet, many texts in other languages include English or Latin names and words. Now you have the opposite issue -- your TTS software needs to switch to English to pronounce these correctly...

And mind you, these are just very simple use cases for TTS. If you delve into use cases for people with limited sight that experience the entire Internet, and all mobile and desktop applications (often having poor localisation) via TTS you see how mono-lingual TTS is mostly useless and would be switched for a robotic old-school TTS in a flash...

> only that but it's also common to have system language set to English

Ask a German whether their system language is English. Ask a French person. I can go on.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#29

Post by iamrobertismo »

singpolyma3 wrote: post #15

Love this. It says MIT license but then readme has a separate section on prohibited use that maybe adds restrictions to make it nonfree? Not sure the legal implications here.

Yeah, I don't understand the point of the prohibited use section at all, seems like unnecessary fluff.

Re: Pocket TTS: A high quality TTS that gives your CPU a voice

#30

Post by phoronixrly »

kamranjon wrote: post #20
phoronixrly wrote: post #12

Earlier quoted context omitted.

I echo this. For a TTS system to be in any way useful outside the tiny population of the world that speaks exclusively English, it must be multilingual and dynamically switch between languages pretty much per word. Cool tech demo though!

That's a pretty crazy requirement for something to be "useful" especially something that runs so efficiently on cpu. Many content creators from non-english speaking countries can benefit from this type of release by translating transcripts of their content to english and then running it through a model like this to dub their videos in a language that can reach many more people.

You mean youtubers? And have to (manually) synchronise the text to their video, and especially when youtube apparently offers voice-voice translation out of the box to my and many others' annoyance?
Post reply on HN