Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Dia, an open-weights TTS model for generating realistic dialogue

github.com

Post reply on HN
Print view View original

131–140 of 202 posts

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#131

Post by JonathanFly »

zhyder wrote: post #52

V v cool: first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc! What're the recommended GPU cloud providers for using such open-weights models?

> first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc!

The old Bark TTS is noisy and often unreliable, but pretty great at coughs, throat clears, and yelling. Even dialogs... sometimes. Same Dia prompt in Bark: https://vocaroo.com/12HsMlm1NGdv

Dia sounds much more clear and reliable, wild what 2 people can do in 3 months.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#132

Post by tough »

dangoodmanUT wrote: post #127
zelphirkalt wrote: post #118

Earlier quoted context omitted.

Why would that be a taboo question to ask? It should be the question we always ask, when presented with a model and in some cases we should probably reject the model, based on that information.

Because generally the person asking this question is trying to cancel the model maker

or by replying you expose yourself to handing -proof- of the origins of the training data set to the copyright owner wanting to sue you next

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#134

Post by daemonologist »

toebee wrote: post #104

We have a ZeroGPU Space provided by HuggingFace up and running! Test it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B

The examples on your site are impressive, but I'm having trouble getting good results on HF - it's generating a lot of near-silence (often nothing but) and when it does produce speech it bears no resemblance to the audio prompt and only produces parts of the text prompt. Would you suggest any adjustments to the default parameters to improve adherence, or might I expect better results running locally? Thanks!

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#135

Post by cchance »

toebee wrote: post #2

Hey HN! We’re Toby and Jay, creators of Dia. Dia is 1.6B parameter open-weights model that generates dialogue directly from a transcript. Unlike TTS models that generate each speaker turn and stitch them together, Dia generates the entire conversation in a single pass. This makes it faster, more natural, and easier to use for dialogue generation. It also supports audio prompts — you can condition the output on a spec…

Its really amazing cant wait to play with it some, the samples are great... but oddly all seem... really fast, like they'd be perfect but they feel like they're playing at 1.2x speed or is that just me?

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#136

Post by cchance »

toebee wrote: post #103
nickthegreek wrote: post #44

Earlier quoted context omitted.

Are there any examples of the audio differences between the this and the larger model?

We're still experimenting, so do not have samples yet from the larger model. All we have is Dia-1.6B at the moment.

I didn't see or missed it are you planning to release the larger model as well?

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#137

Post by cchance »

azinman2 wrote: post #49
Versipelle wrote: post #26

This is really impressive; we're getting close to a dream of mine: the ability to generate proper audiobooks from EPUBs. Not just a robotic single voice for everything, but different, consistent voices for each protagonist, with the LLM analyzing the text to guess which voice to use and add an appropriate tone, much like a voice actor would do. I've tried "EPUB to audiobook" tools, but they are really miles behind wh…

Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

You really think people writing these papers actually have good speaking voices? LOL, theirs a reason not everyone could be an audio book maker or podcaster, a lot of peoples voices suck for audiobooks

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#138

Post by kevmo314 »

moritonal wrote: post #133

Isn't it weird how "We don't have a full list of non-verbal [commands]". Like, I can imagine why, but it's wild we're at a point where we don't know what our code can do.

I have a sneaking suspicion it's because they lifted the model architecture almost directly from Parakeet: https://jordandarefsky.com/blog/2024/parakeet/

Parakeet references WhisperD which is at https://huggingface.co/jordand/whisper-d-v1a and doesn't include a full list of non-speech events that it's been trained with, except "(coughs)" and "(laughs)".

Not saying the authors didn't do anything interesting here. They put in the work to reproduce the blog post and open source it, a praiseworthy achievement in itself, and they even credit Parakeet. But they might not have the list of commands for more straightforward reasons.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#139

Post by toebee »

crooked-v wrote: post #126
toebee wrote: post #98

Earlier quoted context omitted.

Sorry for the confusion. the license is plain Apache 2.0, and we changed the wording to "intended for research and educational use." The point was, users are free to use it for their use cases, just don't do shady stuff with it. Thanks for the feedback :)

So is that actually part of the license (making it non-Apache 2.0), or not?

not part of the license!

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#140

Post by toebee »

kevmo314 wrote: post #138
moritonal wrote: post #133

Isn't it weird how "We don't have a full list of non-verbal [commands]". Like, I can imagine why, but it's wild we're at a point where we don't know what our code can do.

I have a sneaking suspicion it's because they lifted the model architecture almost directly from Parakeet: https://jordandarefsky.com/blog/2024/parakeet/ Parakeet references WhisperD which is at https://huggingface.co/jordand/whisper-d-v1a and doesn't include a full list of non-speech events that it's been trained with, except "(coughs)" and "(laughs)". Not saying the authors didn't do anything interesting here. They…

You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way.

It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.

As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.

Post reply on HN