Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The era of open voice assistants

home-assistant.io

Post reply on HN
Print view View original

181–190 of 290 posts

Re: The era of open voice assistants

#181

Post by alias_neo »

stavros wrote: post #86
fons wrote: post #82

I wonder how this compares to the Respeaker 2 https://wiki.seeedstudio.com/ReSpeaker_Mic_Array_v2.0/ The respeaker has 4 mics and can easily cancel out the noise introduced by a custom external speaker

I don't just want the hardware, I want the software too. I want something that will do STT on my speech, send the text to an API endpoint I control, and be able to either speak the text I give it, or live stream an audio response to the speakers. That's the part I can't do on my own, and then I'll take care of the LLMs myself.

All of these components are available separately or as add-ons for Home Assistant.

I currently do STT with heywillow[0] and an S3-Box-3 which uses an LLM running on a server I have to do incredibly fast, incredibly accurate STT. It uses Coqui XTTS for TTS, with very high quality LLM based voice; you can also clone a voice by supplying it with a few seconds of audio (I tested cloning my own with frightening results).

Playback to a decent speaker can be done in a bunch of ways; I wrote a shim that captures the TTS request to Coqui and forwards it to a Pi based speaker I built, running MPD which then requests the audio from the STT server (Coqui) and plays it back on my higher quality speaker than the crappy ones built in to the voice-input devices.

If you just want to use what's available HA, there's all of the Wyoming stuff, openWakeword (not necessary if you're using this new Voice PE because it does on-device wakeword), Piper for TTS, or MaryTTS (or others) and Whisper (faster-whisper) for STT, or hook in something else you want to use. You can additionally use the Ollama integration to hook it into an Ollama model running on higher end hardware for proper LLM based reasoning.

[0]heywillow.io

Re: The era of open voice assistants

#182

Post by alias_neo »

mkagenius wrote: post #10

Though a separate hardware helps - I believe voice and automation can be integrated more seamlessly to our existing devices (phones/laptops) with high compute built in. Llama and whisper are already public so that should help innovation in this area.

The voice input can really be done however you like, the benefit of a device like the Voice PE is the wake word detection on-device.

I have an office-style desk-phone (SNOM) connected to a SIP server and I can pick the receiver up and talk to the Assistant, but you can plug in any way you like to get the audio to/from HA.

With your phone, wake words are usually locked down by Apple/Google so you can't really have it hands-free, and that's the problem this device is solving; not the audio input itself, but the wake-word/handfree input.

On an Android phone, you can replace the Google Assistant with the Home Assistant one, but you still have to activate it the usual way, press a button or launch the app etc.

Re: The era of open voice assistants

#183

Post by staunton »

nine_k wrote: post #177
pimeys wrote: post #108

Earlier quoted context omitted.

I'm also very excited. I've had some ESP32 microphones before, but they were not really able to understand the wake word, sometimes even when it was quiet and you were sitting next to the mic. This one looks like it can recognize your voice very well, even when music is playing. Because... when it works, it's amazing. You get that Star Trek wake word (KHUM-PUTER!), you can connect your favorite LLM to it (ChatGPT, Cl…

As a side note, it always slightly puzzles me when I see "voice interface" and "private" used together. Maybe it takes living alone to issue voice commands and feel some privacy. (Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties.)

> Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties.

Then why does it puzzle you?

Re: The era of open voice assistants

#184

Post by alias_neo »

singularity2001 wrote: post #77

sorry if this question takes away from the great strives the team went through but wouldn't it be much easier (hardware wise) to jailbreak one of the existing great hardware thingies like Apple HomePod or the Google one or Alexa?

The fact that it hasn't (widely?) been done yet suggests the answer is "no".

The hardware in those devices is generally better, most of them have much better speakers, but they're locked down, the wake-word detection hardware isn't open or accessible so changing it to do what we need would be difficult, and you're just hoping there's a way in.

Existing examples of opening them (as in freedom) replace the PCB entirely, which puts you back to square one of needing open hardware.

This feels like the right approach to me; I've been building my own devices for this purpose with off-the-shelf parts, and designing enclosures, but this is much sleeker; I just hope an add-on or future version comes with much better audio out (speakers) because that's where it and things like it (e.g. the S3-Box-3) are really lacking.

Re: The era of open voice assistants

#185

Post by stavros »

alias_neo wrote: post #181
stavros wrote: post #86

Earlier quoted context omitted.

I don't just want the hardware, I want the software too. I want something that will do STT on my speech, send the text to an API endpoint I control, and be able to either speak the text I give it, or live stream an audio response to the speakers. That's the part I can't do on my own, and then I'll take care of the LLMs myself.

All of these components are available separately or as add-ons for Home Assistant. I currently do STT with heywillow[0] and an S3-Box-3 which uses an LLM running on a server I have to do incredibly fast, incredibly accurate STT. It uses Coqui XTTS for TTS, with very high quality LLM based voice; you can also clone a voice by supplying it with a few seconds of audio (I tested cloning my own with frightening results).…

I do the same, Willow has been unmaintained for close to a year, and calling it "incredibly fast" and "incredibly accurate" tells me that we have very different experiences.

Re: The era of open voice assistants

#186

Post by alias_neo »

joshstrange wrote: post #43
nickthegreek wrote: post #9

And on back order everywhere. I just spent the last 2 weeks getting a esp32-s3-box setup to do this but its lack of audio out really irks me.

And the mic is not all that great either. I have a couple of them but they just weren't reliably picking up my voice and I couldn't hear the reply either (when it did hear me). I figured it would be easy to add a speaker to them but that sent me down a rabbit hole that I gave up on and put them in a drawer. I'll buy this for sure though because when the ESP32 box thing worked it worked really well and I loved being a…

To be fair, the issue with the Box-3 is HA's implementation; I used it with heywillow.io and it was incredible, I could speak to it from another room and it would pick up perfectly.

The audio out is terrible so I wrote a shim-server that captures the request to the TTS server for heywillow and sent it to a speaker I build myself running MPD on a Pi with a nice DAC and have it play the responses instead of the box-3's tiny speaker.

I don't expect the audio-out on this to be much better with its tiny speaker, but at least it has a 3.5mm jack.

I'm going to look into what that Grove port can do too and perhaps build a new speaker "module" that the Voice PE can sit on top of to make it a proper music device.

Re: The era of open voice assistants

#187

Post by jazzyjackson »

moffkalast wrote: post #155
jazzyjackson wrote: post #143

Earlier quoted context omitted.

? I have my house packed to the brim with tplink Wi-Fi smart switches, they work fine. https://www.tp-link.com/us/home-networking/smart-switch/

Ah right I forget I'm talking to Americans on an American site, who all have walls made out of wood and gypsum. Try that with brick and steel reinforced concrete lol.

:)

Re: The era of open voice assistants

#188

Post by alias_neo »

stavros wrote: post #185
alias_neo wrote: post #181

Earlier quoted context omitted.

All of these components are available separately or as add-ons for Home Assistant. I currently do STT with heywillow[0] and an S3-Box-3 which uses an LLM running on a server I have to do incredibly fast, incredibly accurate STT. It uses Coqui XTTS for TTS, with very high quality LLM based voice; you can also clone a voice by supplying it with a few seconds of audio (I tested cloning my own with frightening results).…

I do the same, Willow has been unmaintained for close to a year, and calling it "incredibly fast" and "incredibly accurate" tells me that we have very different experiences.

It's a shame it's been getting no updates, I noticed that, but their secret sauce is all open stuff anyway so just replace them with the upstream components; their box-3 firmware and the application server is really the bit they built (as well as the "correction" service).

If it wasn't fast or accurate for you, what were you running it on? I'm using the large model on a Tesla GPU in a Ryzen 9 server, using the XTTS-2 (Coqui) branch.

The thing about ML based STT/TTS and the reasoning/processing is that you get better performance the more hardware you throw at it; I'm using nearly £4k worth of hardware to do it; is it worth it? No, is it reasonable? Also no, but I already had the hardware and it's doing other things.

I'll switch over to Assist and run Ollama instead now there's some better hardware with on-device wake-word from Nabu.

Re: The era of open voice assistants

#190

Post by TheCoelacanth »

fragmede wrote: post #53
antonyt wrote: post #34

Earlier quoted context omitted.

Yeah, I’m having a hard time imagining how no-wake-word could work in practice.

after setting up the system, if I say "turn the ceiling lights to 20%", who else would be changing the lights? But also, post-fix wake word would also be natural if it was recording all the time. "turn on the lights, Google", for instance

Someone in a TV show that you're watching?
Post reply on HN