Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-TTS family is now open sourced: Voice design, clone, and generation

qwen.ai

Post reply on HN
Print view View original

151–160 of 229 posts

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#151

Post by storystarling »

TheAceOfHearts wrote: post #139
dsrtslnd23 wrote: post #127

Earlier quoted context omitted.

do you have the RTF for the 1080? I am trying to figure out if the 0.6B model is viable for real-time inference on edge devices.

Yeah, it's not great. I wrote a harness that calculates it as: 3.61s Load Time, 38.78s Gen Time, 18.38s Audio Len, RTF 2.111. The Tao Te Ching audiobook came in at 62 mins in length and it ran for 102 mins, which gives an RTF of 1.645. I do get a warning about flash-attn not being installed, which says that it'll slow down inference. I'm not sure if that feature can be supported on the 1080 and I wasn't up for tinker…

An RTF above 1 for just 0.6B parameters suggests the bottleneck isn't the GPU, even on a 1080. The raw compute should be much faster. I'd bet it's mostly CPU overhead or an issue with the serving implementation.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#152

Post by KaoruAoiShiho »

TheAceOfHearts wrote: post #83
KaoruAoiShiho wrote: post #67

Earlier quoted context omitted.

Have you tried specifying the emotion? There's an option to do so and if it's left empty it wouldn't surprise me if it defaulted to rng instead of bland.

For the system prompt I used: > Read this in a calm, clear, and wise audiobook tone. > Do not rush. Allow the meaning to sink in. But maybe I should experiment with something more detailed. Do you have any suggestions?

Something like this:

Character Name: Marcus Cole Voice Profile: A bright, agile male voice with a natural upward lift, delivering lines at a brisk, energetic pace. Pitch leans high with spark, volume projects clearly—near-shouting at peaks—to convey urgency and excitement. Speech flows seamlessly, fluently, each word sharply defined, riding a current of dynamic rhythm. Background: Longtime broadcast booth announcer for national television, specializing in live interstitials and public engagement spots. His voice bridges segments, rallies action, and keeps momentum alive—from voter drives to entertainment news. Presence: Late 50s, neatly groomed, dressed in a crisp shirt under studio lights. Moves with practiced ease, eyes locked on the script, energy coiled and ready. Personality: Energetic, precise, inherently engaging. He doesn’t just read—he propels. Behind the speed is intent: to inform fast, to move people to act. Whether it’s “text VOTE to 5703” or a star-studded tease, he makes it feel immediate, vital.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#153

Post by genewitch »

simonw wrote: post #131

I got this running on macOS using mlx-audio thanks to Prince Canuma: https://x.com/Prince_Canuma/status/2014453857019904423 Here's the script I'm using: https://github.com/simonw/tools/blob/main/python/q3_tts.py You can try it with uv (downloads a 4.5GB model on first run) like this: uv run https://tools.simonwillison.net/python/q3_tts.py \ 'I am a pirate, give me your gold!' \ -i 'gruff voice' -o pirate.wav

If i am ever in the same city as you, i'll buy you dinner. I poked around during my free time today trying to figure out how to run these models, and here is the estimable Simon Willison just presenting it on a platter.

hopefully i can make this work on windows (or linux, i guess).

thanks so much.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#154

Post by aprilthird2021 »

rdtsc wrote: post #48
javier123454321 wrote: post #41

Earlier quoted context omitted.

This is terrifying. With this and z-image-turbo, we've crossed a chasm. And a very deep one. We are currently protected by screens, we can, and should assume everything behind a screen is fake unless rigorously (and systematically, i.e. cryptographically) proven otherwise. We're sleepwalking into this, not enough people know about it.

That was my thought too. You’d have “loved ones” calling with their faces and voices asking for money in some emergency. But you’d also have plausible deniability as anything digital can be brushed off as “that’s not evidence, it could be AI generated”.

> as anything digital can be brushed off as “that’s not evidence, it could be AI generated”.

This won't change anything about Western style courts which have always required an unbroken chain of custody of evidence for evidence to be admissable in court

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#155

Post by genewitch »

indigodaddy wrote: post #144
simonw wrote: post #131

I got this running on macOS using mlx-audio thanks to Prince Canuma: https://x.com/Prince_Canuma/status/2014453857019904423 Here's the script I'm using: https://github.com/simonw/tools/blob/main/python/q3_tts.py You can try it with uv (downloads a 4.5GB model on first run) like this: uv run https://tools.simonwillison.net/python/q3_tts.py \ 'I am a pirate, give me your gold!' \ -i 'gruff voice' -o pirate.wav

Simon how do you think this would perform on CPU only? Lets say threadripper with 20G ram. (Voice cloning in particular)

the old voice cloning and/or TTS models were CPU only, and they weren't realtime, but no worse than 2:1, 30 seconds of audio would take 60 seconds to generate. roughly. in 2021 one-shot TTS/cloning using GPUs was getting there, and that was close enough to realtime; one could, if one was willing to deal with it, wire microphone audio to the model, and speak words, and the model would, in real time, modify the voice. Phil Hendrie is jealous.

anyhow, with faster CPUs and optimizations, you won't be waiting too long. Also 20GB is overkill for an audio model. Only text - LLM - are huge and take infinite memory. SD/FLUX models are under 16GB of ram usage (uh, mine are, at least!), for instance.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#156

Post by genewitch »

TheAceOfHearts wrote: post #139
dsrtslnd23 wrote: post #127

Earlier quoted context omitted.

do you have the RTF for the 1080? I am trying to figure out if the 0.6B model is viable for real-time inference on edge devices.

Yeah, it's not great. I wrote a harness that calculates it as: 3.61s Load Time, 38.78s Gen Time, 18.38s Audio Len, RTF 2.111. The Tao Te Ching audiobook came in at 62 mins in length and it ran for 102 mins, which gives an RTF of 1.645. I do get a warning about flash-attn not being installed, which says that it'll slow down inference. I'm not sure if that feature can be supported on the 1080 and I wasn't up for tinker…

you can install flash attention, et al, but if you're on windows, afaik, you can't use/run/install "triton kernels", which apparently make audio models scream. Whisper complains every time i start it, and it is pretty slow; so i just batch hundreds of audio files on a machine in the corner with a 3060 instead. technically i could batch them on a CPU, too, since i don't particularly care when they finish.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#157

Post by genewitch »

gcr wrote: post #136
imiric wrote: post #129

Earlier quoted context omitted.

Your GitHub profile is... disturbing. 1,354 commits and 464 pull requests in January so far. Regardless of how productive those numbers may seem, that amount of code being published so quickly is concerning, to say the least. It couldn't have possibly been reviewed by a human or properly tested. If this is the future of software development, society is cooked.

You may not like it but this is what a 10x developer looks like. :-)

you may enjoy spaghetti, but will you enjoy 10x spaghetti?

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#158

Post by genewitch »

WarmWash wrote: post #19
throwaw12 wrote: post #6

Qwen team, please please please, release something to outperform and surpass the coding abilities of Opus 4.5. Although I like the model, I don't like the leadership of that company and how close it is, how divisive they're in terms of politics.

The Chinese labs distill the SOTA models to boost the performance of theirs. They are a trailer hooked up (with a 3-6 month long chain) to the trucks pushing the technology forwards. I've yet to see a trailer overtake it's truck. China would need an architectural breakthrough to leap American labs given the huge compute disparity.

can you point me at another free voice cloning / tts model with this fidelity and, i guess prompt adherence?

because i've been on youtube and insta, and believe me, no one else even compares, yet.

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#159

Post by indigodaddy »

andhuman wrote: post #79
indigodaddy wrote: post #8

How does the cloning compare to pocket TTS?

It’s uncanny good. I prefer it to pocket, but then again pocket is much smaller and for realtime streaming.

Ah right I guess I meant for instant which I assume qwen can't do

Re: Qwen3-TTS family is now open sourced: Voice design, clone, and generation

#160

Post by numpad0 »

albertwang wrote: post #2

great news, this looks great! is it just me, or do most of the english audio samples sound like anime voices?

I suspect they might be using voice lines from Chinese gacha games in addition to what clearly sound like VTubers, YouTubers, and Chinese TV documentary narrations. Those games all come with clean monaural CN/JP/EN files consistent in contents across language for all regions, for, an obvious[1] reason.

1: https://old.reddit.com/r/ZenlessZoneZero/comments/1gqmtl1/th...

Post reply on HN