Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatterbox TTS

github.com

Post reply on HN
Print view View original

51–60 of 197 posts

Re: Chatterbox TTS

#51

Post by stevage »

Interesting demo. A few observations, having uploaded a snippet of my own voice, and testing with some of my own text:

- the output had some of the qualities of my voice, but wasn't super similar. (Then again, the fact it could even do this from such a tiny snippet was impressive)

- increasing "CFG/pace" (whatever CFG is) even a little bit often just breaks down into total gibberish

- it was very inconsistent whether it would come out with a kind of British accent or an American one. (My accent is Australian...)

- the emotional exaggeration was interesting, but it seemed to vary a lot exactly what kind of emotion would come out

Re: Chatterbox TTS

#52

Post by ianbicking »

mikepurvis wrote: post #18
ianbicking wrote: post #13

Earlier quoted context omitted.

FWIW in my recent experience I've found LLMs are very good at reading through the transcription errors (I've yet to experiment with giving the LLM alternate transcriptions or confidence levels, but I bet they could make good use of that too)

I was going to say, ideally you’d be able to funnel alternates to the LLM, because it would be vastly better equipped to judge what is a reasonable next word than a purely phonetic model.

If you just give the transcript, and tell the LLM it is a voice transcript with possible errors, then it actually does a great job in most cases. I mostly have problems with mistranscriptions saying something entirely plausible but not at all what I said. Because the STT engine is trying to make a semantically valid transcription it often produces grammatically correct, semantically plausible, and incorrect transcriptions. These really foil the LLM.

Even if you can just mark the text as suspicious I think in an interactive application this would give the LLM enough information to confirm what you were saying when a really critical piece of text is low confidence. The LLM doesn't just know what are the most plausible words and phrases for the user to say, but the LLM can also evaluate if the overall gist is high or low confidence, and if the resulting action is high or low risk.

Re: Chatterbox TTS

#53

Post by 01HNNWZ0MV43FF »

pryelluw wrote: post #20

Silly question, what’s the lowest spec hardware this will run ?

I was going to report how it runs on an old CPU but after fussing with it for about 30 minutes, I can't even get it to run.

Listing the issues in case it helps anyone:

- It doesn't work with Python 3.13, luckily `uv` makes it easy to build a venv with 3.12

- It said numpy 1.26.4 doesn't exist. It definitely does, but `uv pip` was searching for it on the pytorch repo. I passed an `--index-strategy` flag so it would check other repos. This could just be a bug in uv, but when I see "numpy 1.26.4 doesn't exist" and numpy is currently on 2.x, my brain starts to cramp up.

- The `pip install chatterbox-tts` version has a bug in CPU-only mode, so I cloned the Git repo

- The version at the tip of main requires `protobuf-compiler` installed on Debian

- I got a weird CMake error that I can't decipher. I think maybe it's complaining that the Python dev headers are not installed. Why would they be, I'm trying to do inference, not compile Python...

I know anger isn't productive but this is my experience almost any time I'm running Somebody Else's Python Project. Hit an issue, back up, hit another issue, back up, after an hour it still doesn't run.

Re: Chatterbox TTS

#54

Post by russellbeattie »

raincole wrote: post #17
abraxas wrote: post #7

Are these things good enough to narrate a book convincingly or does the voice lose coherence after a few paragraphs being spoken?

Once it's good enough Audible will be flooded with AI-narrated books so we'll know soon. (The only question is whether Amazon would disclose it, ofc)

Audible has already flooded their store with generated audio books. Go to the "Plus Catalog" and it's filled with them. The quality at the moment is complete trash, but I can't imagine it won't get better quickly.

The whole audiobook business will eventually disappear - probably within the decade. There will only be ebooks and on-device AI assistants will read it to you on demand.

I imagine it'll go like this: First pre-generated audiobooks as audio files. Next, online service to generate audio on demand with hyper customizable voices which can be downloaded. Next, a new ebook format which embeds instructions for narration and pronunciation to be read on-device. Finally, AI that's good enough to read it like a storyteller instantly without hints.

Re: Chatterbox TTS

#57

Post by hoherd »

Shopper0552 wrote: post #55

Anyone know a good free open source speech to text? Looking for something for my laptop which is running Fedora KDE plasma.

Whisper has been great for me. I have a single-file uv powered python script that creates SRT files or timestamped text files from media stored on the filesystem. https://github.com/danielhoherd/pub-bin/blob/main/whisper-tr...

Re: Chatterbox TTS

#59

Post by keyle »

pryelluw wrote: post #20

Silly question, what’s the lowest spec hardware this will run ?

It's not a silly question, it's the best question!

If something can be run for free but it's cheaper to rent, it voids the DIY aspect of it.

Re: Chatterbox TTS

#60

Post by echelon »

Mizza wrote: post #8

Demos here: https://resemble-ai.github.io/chatterbox_demopage/ (not mine) This is a good release if they're not too cherry picked! I say this every time it comes up, and it's not as sexy to work on, but in my experiments voice AI is really held back by transcription, not TTS. Unless that's changed recently.

I absolutely ADORE that this has swearing directly in the demo. And from Pulp Fiction, too!

> Any of you fucking pricks move and I'll execute every motherfucking last one of you.

I'm so tired of the boring old "miss daisy" demos.

People in the indie TTS community often use the Navy Seals copypasta [1, 2]. It's refreshing to see Resemble using swear words themselves.

They know how this will be used.

[1] https://en.wikipedia.org/wiki/Copypasta

[2] https://knowyourmeme.com/memes/navy-seal-copypasta

Post reply on HN