Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatterbox TTS

github.com

Post reply on HN
Print view View original

101–110 of 197 posts

Re: Chatterbox TTS

#101

Post by fastball »

echelon wrote: post #76
echelon wrote: post #61

Earlier quoted context omitted.

1. Any non-OpenAI, non-Google, non-ElevenLabs player is going to have to aggressively open source or they'll become 100% irrelevant. The TTS market leaders are obvious and deeply entrenched, and Resemble, Play(HT), et al. have to aggressively cater to developers by offering up their weights [1]. 2. This is CYA for that. Without watermarking, there will be cries from the media about abuse (from anti-AI outfits like 40…

Nevermind, this is just ~3/10 open, or not really open at all [1]: https://github.com/resemble-ai/chatterbox/issues/45#issuecom... > For now, that means we’re not releasing the training code, and fine-tuning will be something we support through our paid API ( https://app.resemble.ai ). This helps us pay the bills and keep pushing out models that (hopefully) benefit everyone. Big bummer here, Resemble. This is not at…

The weights are indeed open (both accessible and licensing-wise): you don't need to put that in square quotes. Training code is not. You can fine-tune the weights yourself with your own training code. Saying that isn't open is like saying ffmpeg isn't open because it doesn't do everything I need it to do and I have to wrap it with own code to achieve my goals.

Re: Chatterbox TTS

#102

Post by bschwindHN »

echelon wrote: post #60
Mizza wrote: post #8

Demos here: https://resemble-ai.github.io/chatterbox_demopage/ (not mine) This is a good release if they're not too cherry picked! I say this every time it comes up, and it's not as sexy to work on, but in my experiments voice AI is really held back by transcription, not TTS. Unless that's changed recently.

I absolutely ADORE that this has swearing directly in the demo. And from Pulp Fiction, too! > Any of you fucking pricks move and I'll execute every motherfucking last one of you. I'm so tired of the boring old "miss daisy" demos. People in the indie TTS community often use the Navy Seals copypasta [1, 2]. It's refreshing to see Resemble using swear words themselves. They know how this will be used. [1] https://en.wik…

Heh, I always type out the first sentence or two of the Navy Seal copypasta when trying out keyboards.

Re: Chatterbox TTS

#104

Post by OfflineSergio »

skygazer wrote: post #77
landl0rd wrote: post #25

Earlier quoted context omitted.

Flip side is a solution where I can have a book without an audiobook auto-generated (or use an existing ebook rather than paying audible $30 for their version) and it's "good enough" is a legit improvement. AI generated isn't as good but it's better than nothing. Also, being able to interrupt and ask for more detail/context would be pretty nice. Like I'm reading some Pynchon and I have to stop sometimes and look up t…

If you're willing to forgo the interactive LLM bit, kokoro-tts (just a script using Kokoro-ONNX) takes epubs and outputs a series of wavs or mp3s that need to be stitched together into chapters or audiobook m4a with some ffmpeg fu. I've listened to several generated audiobooks, and found them pretty good. Some nice generic narration-like prosody. It uses espeak-ng to generate phonemes and passes those to the model to…

I've used this repo and its great. It was one many things that inspired me in building a similar tool. I built https://desktop.with.audio

It was important to me that it be 100% private and local and wanted it to be a one time payment solution. Because it locally process your data it can be a one time payment text to speech app.

If you are interested in creating audiobooks from epubs check this demo: https://www.youtube.com/watch?v=pOHzo6Oq0lQ If you are interested in listening while reading with text highlighting check these demos: - https://www.youtube.com/watch?v=8yJ-lsbzAuw - https://www.youtube.com/watch?v=y8wi4d8xmnw

Re: Chatterbox TTS

#105

Post by satvikpendem »

russellbeattie wrote: post #54
raincole wrote: post #17

Earlier quoted context omitted.

Once it's good enough Audible will be flooded with AI-narrated books so we'll know soon. (The only question is whether Amazon would disclose it, ofc)

Audible has already flooded their store with generated audio books. Go to the "Plus Catalog" and it's filled with them. The quality at the moment is complete trash, but I can't imagine it won't get better quickly. The whole audiobook business will eventually disappear - probably within the decade. There will only be ebooks and on-device AI assistants will read it to you on demand. I imagine it'll go like this: First…

> There will only be ebooks and on-device AI assistants will read it to you on demand.

Honestly I read (or rather, listen to) a lot of books already by getting the epubs onto my phone then using a very basic TTS to read it out. Yes, they're definitely not as lifelike as even the most common AI TTS systems but they're good enough to listen to at high speed. Moon+ Reader is pretty good for Android, not sure about iOS.

Re: Chatterbox TTS

#106

Post by [deleted] »

xnx wrote: post #86
echelon wrote: post #79

Earlier quoted context omitted.

Sadly they don't publish any training or fine tuning code, so this isn't "open" in the way that Flux or Stable Diffusion are "open". If you want better "open" models, these all sound better for zero shot: Zeroshot TTS: MaskGCT, MegaTTS3 Zeroshot VC: Seed-VC, MegaTTS3 Granted, only Seed-VC has training/fine tuning code, but all of these models sound better than Chatterbox. So if you're going to deal with something you…

Great tip. I hadn't heard of MegaTTS3.

[deleted]

Re: Chatterbox TTS

#107

Post by echelon »

fastball wrote: post #101
echelon wrote: post #76

Earlier quoted context omitted.

Nevermind, this is just ~3/10 open, or not really open at all [1]: https://github.com/resemble-ai/chatterbox/issues/45#issuecom... > For now, that means we’re not releasing the training code, and fine-tuning will be something we support through our paid API ( https://app.resemble.ai ). This helps us pay the bills and keep pushing out models that (hopefully) benefit everyone. Big bummer here, Resemble. This is not at…

The weights are indeed open (both accessible and licensing-wise): you don't need to put that in square quotes. Training code is not. You can fine-tune the weights yourself with your own training code. Saying that isn't open is like saying ffmpeg isn't open because it doesn't do everything I need it to do and I have to wrap it with own code to achieve my goals.

Machine learning assets are not binary "open" or "closed". There is a continuum of openness.

To make a really poor analogy, this repo is like a version of Linux that you can't cross-compile or port.

To make another really poor (but fitting) analogy, this is like an "open core" SaaS platform that you know you'll never be able to run the features that matter on your own.

This repo scores really low on the "openness" continuum. In this case, you're very limited in what you can do with Chatterbox TTS. You certainly can't improve it or fit it to your data.

> You can fine-tune the weights yourself with your own training code.

This will never be built by anyone, and they know that. If it could be, they'd provide it themselves.

If you're considering Chatterbox TTS, just use MegaTTS3 [1] instead. It's better by all accounts.

[1] https://github.com/bytedance/MegaTTS3

Re: Chatterbox TTS

#108

Post by racecar789 »

I’d sign up for a service that calls a pharmacy on my behalf to refill prescriptions. In certain situations, pharmacies will not list prescriptions on their websites, even though they have the prescriptions on file, which forces the customer to call by phone — a frustrating process.

I do feel bad for pharmacists, their job is challenging in so many ways.

Re: Chatterbox TTS

#109

Post by soulofmischief »

sovok wrote: post #93
vunderba wrote: post #31

Earlier quoted context omitted.

Pairing speech recognition with a LLM acting as a post-processor is a pretty good approach. I put together a script a while back which converts any passed audio file (wav, mp3, etc.), normalizes the audio, passes it to ggerganov whisper for transcription, and then forwards to an LLM to clean the text. I've used it with a pretty high rate of success on some of my very old and poorly recorded voice dictation recordings…

An LLM step also works pretty well for diarization. You get a transcript with speaker-segmentation (with whisper and pyannote for example), SPEAKER_01 says at some point „Hi I’m Bob. And here’s Alice“, SPEAKER_02 says „Hi Bob“ and now the LLM can infer that SPEAKER_01 = Bob and SPEAKER_02 = Alice.

Yep, my agent i built years ago worked very well with this approach, using a whisper-pyannote combo. The fun part is knowning when to end transcription in noisy environments like a coffee shop.
Post reply on HN