Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

community.home-assistant.io

Post reply on HN
Print view View original

91–100 of 153 posts

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#91

Post by bachittle »

ljclifford wrote: post #67

actually the hardest part of a locally hosted voice assistant isn't the llm. it's making the tts tolerable to actually talk to every day. the core issue is prosody: kokoro and piper are trained on read speech, but conversational responses have shorter breath groups and different stress patterns on function words. that's why numbers, addresses, and hedged phrases sound off even when everything else works. the fix is t…

Coqui TTS is actually deprecated, the company shut down. I have a voice assistant that is using gpt-5.4 and opus 4.6 using the subsidized plans from Codex and Claude Code, and it uses STT and TTS from mlx-audio for those portions to be locally hosted: https://github.com/Blaizzy/mlx-audio

Here are the following models I found work well:

- Qwen ASR and TTS are really good. Qwen ASR is faster than OpenAI Whisper on Apple Silicon from my tests. And the TTS model has voice cloning support so you can give it any voice you want. Qwen ASR is my default.

- Chatterbox Turbo also does voice cloning TTS and is more efficient to run than Qwen TTS. Chatterbox Turbo is my default.

- Kitten TTS is good as a small model, better than Kokoro

- Soprano TTS is surprisingly really good for a small model, but it has glitches that prevent it from being my default

But overall the mlx-audio library makes it really easy to try different models and see which ones I like.

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#92

Post by bluGill »

ffsm8 wrote: post #82
nickthegreek wrote: post #77

Earlier quoted context omitted.

100% agree. I dont want a Yes, Got it, Will do or even worse, I have turned on the Bedroom Light. I want soft success ding or a low failure boop.

why would you want an audio notification for a light? it either turns on and it worked or it doesnt turn on. i see no value in having a ding or anything of the kind if i imagine constant dinging whenever i enter a room and the motion sensor toggles the light innit i'd go mad

The biggest use for me is 'guests will be here soon, turn on the lights in front of the shed where they will park', then latter when they are gone turn them off. I can't see the lights from the house and the logical place for a switch isn't in the house. Where I can see the lights a manual switch is better. I don't have most of my lights automated. The ones that are, are that way because I can't see them from where I'd want to check and control them

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#93

Post by bluGill »

vineyardmike wrote: post #49
ethagnawl wrote: post #41

Earlier quoted context omitted.

What's been surprising in my experience regarding the wake word is that it recognizes me (adult male) saying the wake word ~95% of the time. However, it only registers the rest of my family (women and children) ~30% of the time.

I have no firsthand knowledge, but I’d strongly bet that the home-assistant effort to donate training data is mostly get adult males, and nearly zero children.

I remember when those systems first started collecting data they were worried kids wouldn't be handled - but they didn't know how to handle the privacy issuses with recording kids so discouraged it. Women being missed is not a surprise - but not anticipated.

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#94

Post by buildsjets »

ljclifford wrote: post #67

actually the hardest part of a locally hosted voice assistant isn't the llm. it's making the tts tolerable to actually talk to every day. the core issue is prosody: kokoro and piper are trained on read speech, but conversational responses have shorter breath groups and different stress patterns on function words. that's why numbers, addresses, and hedged phrases sound off even when everything else works. the fix is t…

Can you make it sound just like Titus Moody? I want to hear your voice assistant say "No sir, I don't hold with furniture that talks."

https://www.youtube.com/watch?v=BIjjDC3tFfU

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#95

Post by quickthoughts »

ljclifford wrote: post #67

actually the hardest part of a locally hosted voice assistant isn't the llm. it's making the tts tolerable to actually talk to every day. the core issue is prosody: kokoro and piper are trained on read speech, but conversational responses have shorter breath groups and different stress patterns on function words. that's why numbers, addresses, and hedged phrases sound off even when everything else works. the fix is t…

I've been working on the flip side of this with ASR models, but the problem space is the same, conversational/real-world data is needed. Whisper often mistook actual words I say and hallucinate all the time when speaking technical jargon. The solution is to fine-tuning whisper with my own data. Hardest part imo was getting the actual data, which in turn got me to build listenr (https://github.com/rebreda/listenr).It's an always-on VAD-based audio dataset builder. Could be used for building conversational/real-world voice datasets for TTS models too?

After getting it working i was get motivation to actually able to build out the full fine-tuning pipeline. I wrote a little post about it all https://quickthoughts.ca/posts/listenr-asr-training-data-pro...

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#96

Post by Obscurity4340 »

tkems wrote: post #38

One that I have been experimenting with is using analog phones (including rotary ones!) to act as the satellites. I live in an older home and have phone jacks in most of the rooms already so I only had to use a single analog telephone adapter. [0] The downside is I don't have wake word support, but it makes it more private and I don't find myself missing my smart speakers that much. At some point I would like to also…

I wish I was remotely closer to being this kind of hacker :(

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#97

Post by system2 »

quirk wrote: post #87

The best fix I've made to any voice-mode AI is giving it a "done" word. So it has to listen for "pineapple" before it's allowed to process what I said. Just like radio comms (over and out).

I think you will get tired of saying pineapple.

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#98

Post by spacecrafter3d »

croes wrote: post #54
_spduchamp wrote: post #18

Earlier quoted context omitted.

How about a button? I'd prefer to physically press a button on an intercom box than having something churning away constantly processing sound.

Time for a real life Star Trek comm badge

We thought so too ;) https://www.linkedin.com/posts/y-combinator_button-computer-...

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#99

Post by m463 »

ffsm8 wrote: post #82
nickthegreek wrote: post #77

Earlier quoted context omitted.

100% agree. I dont want a Yes, Got it, Will do or even worse, I have turned on the Bedroom Light. I want soft success ding or a low failure boop.

why would you want an audio notification for a light? it either turns on and it worked or it doesnt turn on. i see no value in having a ding or anything of the kind if i imagine constant dinging whenever i enter a room and the motion sensor toggles the light innit i'd go mad

i thought it was specifically when using voice - ack/nack

but it might be preference... some people like clicky blue keys, some like silent red keys on their keyboard for example.

Re: My Journey to a reliable and enjoyable locally hosted voice assistant (2025)

#100

Post by Crespyl »

robotswantdata wrote: post #66
hamdingers wrote: post #10

If you're less concerned about privacy, I use Gemini 2.5 Flash for this and it's exceptionally good and fast as a HA assistant while being much cheaper than the electricity that would be needed to keep a 3090 awake. The thing that kills this for me (and they even mentioned it) is wake word detection. I have both the HA voice preview and FPH Satellite1 devices, plus have experimented with a few other options like a Ra…

What about your wifi APs sensing which room you are in, with your choice of hilarious dance moves as the trigger ? Funky chicken for Gemini Penguin dance for OpenAI Claude?

> Claude?

The Zoidberg Shuffle?

Post reply on HN