Show HN: Dia, an open-weights TTS model for generating realistic dialogue • Hacker News Board

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#121

Post by toebee » Tue, Apr 22, 2025, 1:20 AM UTC

jokethrowaway wrote: post #67

Looking forward to try. My current go-to solution is E5-F2 (great cloning, decent delivery, ok audio quality, a lot of incoherence here and there forcing you to do multiple generations). I've just been massively disappointed by Sesame's CSM: on their gradio on the website it was generating flawless dialogs with amazing voice cloning. When running it local the voice cloning performance is awful.

Thanks for the interest! We also enjoyed using E5-F2 :) You can try it now on HF Spaces: https://huggingface.co/spaces/nari-labs/Dia-1.6B

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#122

Post by toebee » Tue, Apr 22, 2025, 1:21 AM UTC

zhyder wrote: post #52

V v cool: first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc! What're the recommended GPU cloud providers for using such open-weights models?

Thanks you!! We personally used Quickpod and Runpod the most. But you can try it now on HF Spaces without spinning up GPUs yourself!

https://huggingface.co/spaces/nari-labs/Dia-1.6B

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#123

Post by qwertytyyuu » Tue, Apr 22, 2025, 1:22 AM UTC

That Sesame CSM-1B voice sounds sooo done with life, haha.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#124

Post by toebee » Tue, Apr 22, 2025, 1:22 AM UTC

isoprophlex wrote: post #32

Incredible quality demo samples, well done. How's the performance for multilingual generation?

Thank you for the kind words! We only support English at the moment.. Hope to add more languages in the future.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#125

Post by emmelaich » Tue, Apr 22, 2025, 1:25 AM UTC

notdian wrote: post #29

made a small change and got it running on M2 Pro 16GB Macbook pro, the quality is amazing. https://github.com/nari-labs/dia/pull/4

Thanks, works well but slowly on a Mac Air M3 with 24gb. Will have to try it again after freeing up more ram as it was doing a bit of swapping with Chrome running too.

(later). It did nicely for the default example text but just made weird sounds for a "hello all" prompt. And took longer?!

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#126

Post by crooked-v » Tue, Apr 22, 2025, 1:30 AM UTC

toebee wrote: post #98

rustc wrote: post #47

Is this Apache licensed or a custom one? The README contains this: > This project is licensed under the Apache License 2.0 - see the LICENSE file for details. > This project offers a high-fidelity speech generation model *intended solely for research and educational use*. The following uses are strictly forbidden: > Identity Misuse: Do not produce audio resembling real individuals without permission. > ... Specifical…

Sorry for the confusion. the license is plain Apache 2.0, and we changed the wording to "intended for research and educational use." The point was, users are free to use it for their use cases, just don't do shady stuff with it. Thanks for the feedback :)

So is that actually part of the license (making it non-Apache 2.0), or not?

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#127

Post by dangoodmanUT » Tue, Apr 22, 2025, 1:35 AM UTC

zelphirkalt wrote: post #118

dangoodmanUT wrote: post #93

Earlier quoted context omitted.

I know it’s taboo to ask, but I must: where’s the dataset from? Very eager to play around with audio models myself, but I find existing datasets limiting

Why would that be a taboo question to ask? It should be the question we always ask, when presented with a model and in some cases we should probably reject the model, based on that information.

Because generally the person asking this question is trying to cancel the model maker

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#128

Post by karimf » Tue, Apr 22, 2025, 1:42 AM UTC

toebee wrote: post #2

Hey HN! We’re Toby and Jay, creators of Dia. Dia is 1.6B parameter open-weights model that generates dialogue directly from a transcript. Unlike TTS models that generate each speaker turn and stitch them together, Dia generates the entire conversation in a single pass. This makes it faster, more natural, and easier to use for dialogue generation. It also supports audio prompts — you can condition the output on a spec…

This is super awesome. Several questions.

1. What GPU did you use to train the model? I'd love to train a model like this, but currently, I only have a 16GB MacBook. Thinking about buying a 5090 if it's worth.

2. Is it possible to use this for real time audio generation, similar to the demo on the Sesame website?

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#129

Post by sroussey » Tue, Apr 22, 2025, 1:57 AM UTC

Can we get this working in the browser a la ONNX or similar?

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#130

Post by kreelman » Tue, Apr 22, 2025, 2:15 AM UTC

hemloc_io wrote: post #41

Very cool! Insane how much low hanging fruit there is for Audio models right now. A team of two picking things up over a few months can build something that still competes with large players with tons of funding

This is amazing. Is it possible to build in a chosen voice, a bit like Eleven Labs does? ...This may be on the git summary, being lazy and asking anyway :=) Thanks for your work.