Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Dia, an open-weights TTS model for generating realistic dialogue

github.com

Post reply on HN
Print view View original

61–70 of 202 posts

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#61

Post by DoctorOW »

jinay wrote: post #48
tyrauber wrote: post #28

Hey, do yourself a favor and listen to the fun example: > [S1] Oh fire! Oh my goodness! What's the procedure? What to we do people? The smoke could be coming through an air duct! Seriously impressive. Wish I could direct link the audio. Kudos to the Dia team.

For anyone who wants to listen, it's on this page: https://yummy-fir-7a4.notion.site/dia

A little overacted, it reminds me of the voice acting in those flash cartoons you'd see in the early days of YouTube. That's not to say it isn't good work, it still sounds remarkably human. Just silly humans :)

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#63

Post by senordevnyc »

azinman2 wrote: post #49
Versipelle wrote: post #26

This is really impressive; we're getting close to a dream of mine: the ability to generate proper audiobooks from EPUBs. Not just a robotic single voice for everything, but different, consistent voices for each protagonist, with the LLM analyzing the text to guess which voice to use and add an appropriate tone, much like a voice actor would do. I've tried "EPUB to audiobook" tools, but they are really miles behind wh…

Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

Honestly, I’d say that’s true only for the author. Anyone else is just going to be interpreting the words to understand how to best convey the character / emotion / situation / etc., just like an AI will have to do. If an AI can do that more effectively than a human, why not?

The author could be better, because they at least have other info beyond the text to rely on, they can go off-script or add little details, etc.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#64

Post by fwip »

nojs wrote: post #60
tyrauber wrote: post #28

Hey, do yourself a favor and listen to the fun example: > [S1] Oh fire! Oh my goodness! What's the procedure? What to we do people? The smoke could be coming through an air duct! Seriously impressive. Wish I could direct link the audio. Kudos to the Dia team.

This is so good. Reminds me of The Office. I love how bad the other examples are.

The text is lifted from a scene in The Office: https://youtu.be/gO8N3L_aERg?si=y7PggNrKlVQm0qyX&t=82

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#66

Post by Havoc »

Sounds really good & human! Got a fair bit of unexpected artifacts though. e.g. 3 seconds hissing noise before dialogue. And music in background when I added (happy) in an attempt to control tone. Also don't understand how to control the S1 and S2 speakers...is it just random based on temp?

> TODO Docker support

Got this adapted pretty easily. Just latest nvidia cuda container, throw python and modules on it and change server to serve on 0.0.0.0. Does mean it pulls the model every time on startup though which isn't ideal

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#67

Post by jokethrowaway »

Looking forward to try. My current go-to solution is E5-F2 (great cloning, decent delivery, ok audio quality, a lot of incoherence here and there forcing you to do multiple generations).

I've just been massively disappointed by Sesame's CSM: on their gradio on the website it was generating flawless dialogs with amazing voice cloning. When running it local the voice cloning performance is awful.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#68

Post by DrSiemer »

senordevnyc wrote: post #63
azinman2 wrote: post #49

Earlier quoted context omitted.

Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

Honestly, I’d say that’s true only for the author. Anyone else is just going to be interpreting the words to understand how to best convey the character / emotion / situation / etc., just like an AI will have to do. If an AI can do that more effectively than a human, why not? The author could be better, because they at least have other info beyond the text to rely on, they can go off-script or add little details, etc…

As somebody who has listened to hundreds of audiobooks, I can tell you authors are generally not the best choice to voice their own work. They may know every intent, but they are writers, not actors.

The most skilled readers will make you want to read books _just because they narrated them_. They add a unique quality to the story, that you do not get from reading yourself or from watching a video adaptation.

Currently I'm in The Age of Madness, read by Steven Pacey. He's fantastic. The late Roy Dotrice is worth a mention as well, for voicing Game of Thrones and claiming the Guinness world record for most distinct voices (224) in one series.

It will be awesome if we can create readings automatically, but it will be a while before TTS can compete with the best readers out there.

Re: Show HN: Dia, an open-weights TTS model for generating realistic dialogue

#70

Post by Versipelle »

azinman2 wrote: post #49
Versipelle wrote: post #26

This is really impressive; we're getting close to a dream of mine: the ability to generate proper audiobooks from EPUBs. Not just a robotic single voice for everything, but different, consistent voices for each protagonist, with the LLM analyzing the text to guess which voice to use and add an appropriate tone, much like a voice actor would do. I've tried "EPUB to audiobook" tools, but they are really miles behind wh…

Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

> Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

Of course, but it's not always available.

For example, I would love an audiobook for Stanisław Lem's "The Invincible," as I just finished its video game adaptation, yet it simply doesn't exist in my native language.

It's quite seldom that the author narrates the audiobooks I listen to, and sometimes the narrator does a horrible job, butchering the characters with exaggerated tones.

Post reply on HN