Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatterbox TTS

github.com

Post reply on HN
Print view View original

41–50 of 197 posts

Re: Chatterbox TTS

#41

Post by throwawaymaths »

ianbicking wrote: post #13
Mizza wrote: post #8

Demos here: https://resemble-ai.github.io/chatterbox_demopage/ (not mine) This is a good release if they're not too cherry picked! I say this every time it comes up, and it's not as sexy to work on, but in my experiments voice AI is really held back by transcription, not TTS. Unless that's changed recently.

FWIW in my recent experience I've found LLMs are very good at reading through the transcription errors (I've yet to experiment with giving the LLM alternate transcriptions or confidence levels, but I bet they could make good use of that too)

do you know if any current locally hostable public transcribers are good at diarization? for some tasks having even crude diarization would improve QOL by a huge factor. i was looking at a whisper diarization python package for a bit but it was a bitch to deploy.

Re: Chatterbox TTS

#42

Post by iainmerrick »

throwawaymaths wrote: post #41
ianbicking wrote: post #13

Earlier quoted context omitted.

FWIW in my recent experience I've found LLMs are very good at reading through the transcription errors (I've yet to experiment with giving the LLM alternate transcriptions or confidence levels, but I bet they could make good use of that too)

do you know if any current locally hostable public transcribers are good at diarization? for some tasks having even crude diarization would improve QOL by a huge factor. i was looking at a whisper diarization python package for a bit but it was a bitch to deploy.

Deepgram does it.

Re: Chatterbox TTS

#43

Post by vunderba »

Tokumei-no-hito wrote: post #36
vunderba wrote: post #31

Earlier quoted context omitted.

Pairing speech recognition with a LLM acting as a post-processor is a pretty good approach. I put together a script a while back which converts any passed audio file (wav, mp3, etc.), normalizes the audio, passes it to ggerganov whisper for transcription, and then forwards to an LLM to clean the text. I've used it with a pretty high rate of success on some of my very old and poorly recorded voice dictation recordings…

thanks for sharing. are some local models better than others? can small models work well or do you want 8B+?

So in my experience smaller models tend to produce worse results BUT I actually got really good transcription cleanup with CoT (Chain of Thought models) like Qwen even quantized down to 8b.

Re: Chatterbox TTS

#44

Post by throwawaymaths »

iainmerrick wrote: post #42
throwawaymaths wrote: post #41

Earlier quoted context omitted.

do you know if any current locally hostable public transcribers are good at diarization? for some tasks having even crude diarization would improve QOL by a huge factor. i was looking at a whisper diarization python package for a bit but it was a bitch to deploy.

Deepgram does it.

sorry i meant locally hostable public. ill edit parent.

Re: Chatterbox TTS

#45

Post by vunderba »

teraflop wrote: post #35

> Every audio file generated by Chatterbox includes Resemble AI's Perth (Perceptual Threshold) Watermarker - imperceptible neural watermarks that survive MP3 compression, audio editing, and common manipulations while maintaining nearly 100% detection accuracy. Am I misunderstanding, or can you trivially disable the watermark by simply commenting out the call to the apply_watermark function in tts.py? https://github.c…

Yeah, there's even a flag to turn it off in the parser `--no-watermark`. I assumed they added it for downstream users pulling it in as a "feature" for their larger product.

Re: Chatterbox TTS

#46

Post by BoorishBears »

raincole wrote: post #17
abraxas wrote: post #7

Are these things good enough to narrate a book convincingly or does the voice lose coherence after a few paragraphs being spoken?

Once it's good enough Audible will be flooded with AI-narrated books so we'll know soon. (The only question is whether Amazon would disclose it, ofc)

I think you're a bit behind on it: https://www.audible.com/about/newsroom/audible-expands-catal...

Re: Chatterbox TTS

#47

Post by ineedasername »

The emotional exaggeration is interesting, though I don't think I've come across anything quite so versatile and easy to "sculpt" as Elevenlabs and it's ability to generate a voice on the basis of a description of how you want the voice to sound. SparkTTS allows some additional parameters, and it's project on GitHub has placeholders in its code that indicate the model might be refined for more fine grained emotional control. As it is, I've had some success with it and other models by trying to influence prosody and tonality with some heavy handed queues in the text, which can then be used with VC to get closer to desired results, but it's a much more cumbersome process than Eleven.

Re: Chatterbox TTS

#48

Post by causality0 »

Anyone know how this compares to Kokoro? I've found Kokoro very useful for generating audiobook but it almost always pronounces words with paired vowels incorrectly. Daisy becomes die-zee, leave becomes lay-ve, etc.

Re: Chatterbox TTS

#49

Post by junon »

lukax wrote: post #40
Mizza wrote: post #8

Demos here: https://resemble-ai.github.io/chatterbox_demopage/ (not mine) This is a good release if they're not too cherry picked! I say this every time it comes up, and it's not as sexy to work on, but in my experiments voice AI is really held back by transcription, not TTS. Unless that's changed recently.

[flagged]

You should really disclaim that you're affiliated.

https://news.ycombinator.com/item?id=41866830

Re: Chatterbox TTS

#50

Post by stevage »

byteknight wrote: post #12
j2kun wrote: post #11

They should put the meaning of "TTS" in the readme somewhere, probably near the top. Or their website.

TTS is a very common initialism for Text-to-Speech going back to at least the 90s.

Yeah, it's a very common initialism for people who work in the space, and have some context.
Post reply on HN