Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Willow – Open-source privacy-focused voice assistant hardware

github.com

Post reply on HN
Print view View original

61–70 of 141 posts

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#61

Post by woodson »

kkielhofner wrote: post #33
glenngillen wrote: post #29

Earlier quoted context omitted.

What’s the latency on inference on a rasbpi (I assume it’s not running it direct on the device)? I think I read previously that it was up to 7 secs, and if you wanted sub-second you’d need an i5.

Willow supports the Espressif ESP SR speech recognition framework to do completely on device speech recognition for up to 400 commands. When configured, we pull light and switch entities from home assistant and build the grammar to turn them on and off. There's no reason it has to be limited to that, we just need to do some extra work for better dynamic configuration and tighter integration with Home Assistant to all…

Have you considered K2/Sherpa for ASR instead of ctranslate2/faster-whisper? It’s much better suited for streaming ASR (whisper transcribes 30 sec chunks, no streaming). They’re also working on adding context biasing using Aho-Corasick automata, to handle dynamic recognition of eg. contact list entries or music library titles (https://github.com/k2-fsa/icefall/pull/1038).

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#62

Post by kkielhofner »

yoavm wrote: post #59

This is wonderful. I would love to replace my stupid Google Home Minis with this if I can actually get the hardware for $50. The Mycroft device is like $400 so I didn't even consider it, and I never understood why it had to be so expensive. I don't even need a screen - just a microphone. Will definitely give this a shot!

Thank you!

Yes, this is why we went through the pain of doing what we're able to do with this hardware.

Even in this initial release it's competitive with Echo, etc even on cost.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#64

Post by kkielhofner »

java_beyb wrote: post #60
kkielhofner wrote: post #28

Earlier quoted context omitted.

We can do either. For "basic" command recognition the ESP SR (speech recognition) library supports up to 400 defined speech commands that run completely on the device. For most people this is plenty to control devices around the home, etc. Because it is all local it's extremely fast - as I said in another comment pushing "Did that really just happen?" fast. However, for cases where someone wants to be able to throw a…

first, good initiative! thanks for sharing. i think you gotta be more diligent and careful with the problem statement. checking the weather in Sofia, Bulgaria requires cloud, current information. it's not "random speech". ESP SR capability issues don't mean that you cannot process it locally. the comment was on "voice processing" i.e. sending speech to the cloud, not sending a call request to get the weather informat…

Thanks!

There are at least two things here:

1) The ability to do speech to text on random speech. I'm going to stick by that description :). If you've ever watched a little kid play with Alexa it's definitely what you would call "random speech" haha!

2) The ability to satisfy the request (intent) of the text output. Up to and including current information via API, etc.

Our soon to be released highly optimized open source inference server uses Whisper and is ridiculously fast and accurate. Based on our testing with nieces and nephews we have "random speech" covered :). Our inference server also supports LLaMA, Vicuna, etc and can chain together STT -> LLM/API/etc -> TTS - with the output simply played over the Willow speaker and/or displayed on the LCD.

Our goal is to make a Willow Home Assistant component that assists with #2. There are plenty of HA integrations and components to do things like get weather in real time, in addition to satisfying user intent recognition. They have an entire platform for it[0]. Additionally, we will make our inference server implementation (that does truly unique things for Willow) available as just another TTS/STT integration option on top of the implementations they already support so you can use whatever you want, or send the audio output after wake to whatever you want like Vosk, Cheetah, etc, etc.

[0] - https://developers.home-assistant.io/docs/intent_index/

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#65

Post by RileyJames »

I’ve been living in a house for the past few months with a google assistant. I only use it to put on music, but I have noticed I play more music due to the ease of putting it on.

But I hate the privacy invasion aspect. I’m definitely in the market for something like this. And this one looks great.

Additionally, I’ve noticed that the google voice assistant (connected to Spotify) doesn’t keep playing the albums I ask for.

It states it’s playing the album. But after 4/5 songs it starts playing different songs, or different artists.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#66

Post by kkielhofner »

woodson wrote: post #61
kkielhofner wrote: post #33

Earlier quoted context omitted.

Willow supports the Espressif ESP SR speech recognition framework to do completely on device speech recognition for up to 400 commands. When configured, we pull light and switch entities from home assistant and build the grammar to turn them on and off. There's no reason it has to be limited to that, we just need to do some extra work for better dynamic configuration and tighter integration with Home Assistant to all…

Have you considered K2/Sherpa for ASR instead of ctranslate2/faster-whisper? It’s much better suited for streaming ASR (whisper transcribes 30 sec chunks, no streaming). They’re also working on adding context biasing using Aho-Corasick automata, to handle dynamic recognition of eg. contact list entries or music library titles ( https://github.com/k2-fsa/icefall/pull/1038 ).

Whisper, per the model, does 30 second chunks and doesn't support "streaming" by the strict definition.

You'll be able to see when we release our inference server implementation next week that it's more than a version of "realtime" enough to fool nearly anyone, especially with an application like this where you aren't looking for model output in real time. You're streaming speech, buffering on the server, waiting for the end of voice activity detection, running Whisper, taking the transcription, and doing something with it. Other than a cool demo I'm not really sure what streaming ASR output provides but that's probably lack of imagination on my part :).

That said, these are great pointers and we're certainly not opposed to it! At the end of the day Willow does the "work on the ground" of detecting wake word, getting clean audio, and streaming the audio. Where it goes and what happens then is up to you! There's no reason at all we couldn't support streaming ASR output.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#67

Post by kkielhofner »

RileyJames wrote: post #65

I’ve been living in a house for the past few months with a google assistant. I only use it to put on music, but I have noticed I play more music due to the ease of putting it on. But I hate the privacy invasion aspect. I’m definitely in the market for something like this. And this one looks great. Additionally, I’ve noticed that the google voice assistant (connected to Spotify) doesn’t keep playing the albums I ask f…

Music output is "on the list".

Biggest fundamental issue is the speaker built in the ESP BOX is optimized for speech and not going to impress anyone playing music.

That said, the ESP BOX (of course) supports bluetooth so we can definitely pair with a speaker you bring.

Willow is the first of it's kind that I'm aware of to enable this kind of functionality at anything close to this price point in the open source ecosystem. Either we (or someone else) is likely going to manufacture an improved ESP BOX with market competitive speakers built-in for music playback.

Then it's "just" a matter of actually getting the music audio but we'll figure that out ;).

Post reply on HN