Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Willow – Open-source privacy-focused voice assistant hardware

github.com

Post reply on HN
Print view View original

21–30 of 141 posts

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#21

Post by kkielhofner »

tikkun wrote: post #12
kkielhofner wrote: post #7

Earlier quoted context omitted.

Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…

Nice! How's the speech recognition accuracy and response latency?

Thanks!

Faster than Alexa (and only going to get faster)[0].

Between the far-field speech optimizations provided by the ESP BOX and Espressif frameworks and our inference server (open sourcing next week) using Whisper, and our unique streaming format we've found it to be comparable in terms of quality to Alexa/Echo even with background noise and at distances of up to 30 feet.

[0] - https://www.youtube.com/watch?v=8ETQaLfoImc

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#22

Post by tikkun »

kkielhofner wrote: post #21
tikkun wrote: post #12

Earlier quoted context omitted.

Nice! How's the speech recognition accuracy and response latency?

Thanks! Faster than Alexa (and only going to get faster)[0]. Between the far-field speech optimizations provided by the ESP BOX and Espressif frameworks and our inference server (open sourcing next week) using Whisper, and our unique streaming format we've found it to be comparable in terms of quality to Alexa/Echo even with background noise and at distances of up to 30 feet. [0] - https://www.youtube.com/watch?v=8ET…

That's really nice - and thanks for including the demo link too, impressive!

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#23

Post by kkielhofner »

michaelmior wrote: post #17
kkielhofner wrote: post #7

Earlier quoted context omitted.

Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…

> the ESP BOX itself can recognize up to 400 commands directly on the device. That's really cool! Does this mean 400 specific commands, e.g. "turn on the living room lights" or 400 commands that can be applied to different targets, e.g. "turn on the X lights" where X is some light. (400 actually feels like it would be enough to speed up the vast majority of interactions either way, but I'm curious :)

400 commands where "turn on X" is one and "turn off X" is two.

With Home Assistant this means turning on and off two hundred entities. We currently pull light and switch entities from Home Assistant and build the local Multinet speech grammar.

We have goals for better dynamic and adaptive configuration of Willow and part of that is using a Willow Home Assistant component with user configuration inthe HA dashboard, etc to easily select entities, define commands, etc and dynamically update all associated Willow devices.

We feel that with this 400 commands is enough to be practical and useful. Additionally, because the Multinet model returns probability on match to command "fuzzy matching" actually works quite well where "light", "lights", and slightly mis-worded commands still match correctly.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#24

Post by kkielhofner »

WaitWaitWha wrote: post #16

Congratulations! This is great news! I do not see anything posted on the Home Assistant (HA) Community forums. > Configuring and building Willow for the ESP BOX is a multi-step process. We're working on improving that but for now... This is crucial as your "competitors" are ready out of the box. I believe HA can be a Google/Alexa alternative to the masses only if the "out-of-the-box" experience is comparable to the c…

Thanks!

HN was my first stop (of course) - I'll be heading over there shortly to post.

Oh yeah, we're well aware of how much of a "pain" getting Willow going can be. I don't like it (at all).

That said, you configure and build once for your environment and then get a .bin that can be flashed to the ESP BOX with anything that does ESP flashing (like various web interfaces, etc) or you can re-run the flash command across X devices. So even now, in this early stage, it's at least only painful once ;).

Down the road we want to have a Willow Home Assistant component that does everything inside of the HA dashboard so users (like esphome, maybe even using esphome) can point-click-configure-flash entirely from the HA dashboard. Not to mention ongoing dynamic configuration, over the air updates, etc.

I talk about all of this on our wiki[0].

[0] - https://github.com/toverainc/willow/wiki/Home-Assistant

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#26

Post by api »

I love seeing lots of practical refutations of the "we have to do the voice processing in the cloud for performance" rationales peddled by the various home 1984 surveillance box vendors.

It's actually faster to do it locally. They want it tethered to the cloud for surveillance.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#27

Post by kkielhofner »

tikkun wrote: post #22
kkielhofner wrote: post #21

Earlier quoted context omitted.

Thanks! Faster than Alexa (and only going to get faster)[0]. Between the far-field speech optimizations provided by the ESP BOX and Espressif frameworks and our inference server (open sourcing next week) using Whisper, and our unique streaming format we've found it to be comparable in terms of quality to Alexa/Echo even with background noise and at distances of up to 30 feet. [0] - https://www.youtube.com/watch?v=8ET…

That's really nice - and thanks for including the demo link too, impressive!

Thanks again!

Not only are we working on improving performance with the inference server, local on device command recognition is extremely fast. Like "did that really just happen?" fast.

In my local setup when using locally-controlled Wemo switches I swear the latency with local devices is around 300ms or so.

I should make another demo video with that...

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#28

Post by kkielhofner »

api wrote: post #26

I love seeing lots of practical refutations of the "we have to do the voice processing in the cloud for performance" rationales peddled by the various home 1984 surveillance box vendors. It's actually faster to do it locally. They want it tethered to the cloud for surveillance.

We can do either.

For "basic" command recognition the ESP SR (speech recognition) library supports up to 400 defined speech commands that run completely on the device. For most people this is plenty to control devices around the home, etc. Because it is all local it's extremely fast - as I said in another comment pushing "Did that really just happen?" fast.

However, for cases where someone wants to be able to throw any kind of random speech at it "Hey Willow what is the weather in Sofia, Bulgaria?" that's probably beyond the fundamental capabilities of a device with enclosure, display, mics, etc that sells for $50.

That's why we plan to support any of the STT/TTS modules provided by Home Assistant to run on local Raspberry Pis or wherever they host HA. Additionally, we're open sourcing our extremely fast highly optimized Whisper/LLM/TTS inference server next week so people can self host that wherever they want.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#29

Post by glenngillen »

kkielhofner wrote: post #19
edf13 wrote: post #11

Sounds interesting - one question I have is about the mic array... Isn't this one of the supposed benefits of a physical Alexa device, and rumored to be sold at a loss because of the quality. How does the esp-box compare? E.g. in a noisy environemnt, tv in the background, kids and dogs running around?

The ESP BOX has an acoustically optimized enclosure with dual microphones for noise cancelation, separation, etc. Between that and the Espressif AFE (audio frontend interface) doing a bunch of DSP "stuff" in our testing it does remarkably well in noisy environments and far-field (25-30 feet) use cases. Our inference server implementation (open source, releasing next week) uses a highly performance optimized Whisper w…

What’s the latency on inference on a rasbpi (I assume it’s not running it direct on the device)? I think I read previously that it was up to 7 secs, and if you wanted sub-second you’d need an i5.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#30

Post by kkielhofner »

jzebedee wrote: post #13
Brendinooo wrote: post #4

Nice. Any bits of Mycroft in here? That project just imploded and I’m still sad about it.

The "TTS Output" and "Audio on device" sections make it seem like there is no spoken output, only status beeps. A former Mycroft dev, Michael Hansen[1], is still building several year-of-the-voice projects after he was let go. I'm especially excited about Piper[2], which is a C++/py alternative to Mimic3. [1] https://github.com/synesthesiam [2] https://github.com/rhasspy/piper

We plan to make a Home Assistant Willow component to use any of their supported TTS modules to play speech output on device. We just didn't get to it yet.

Our inference server (open source, releasing next week) has highly optimized Whisper, LLaMA/Vicuna/etc, text to speech, etc implementations as well.

It's actually not that hard on the device - if the response from the HA component has audio, play it.

We just don't have the HA component yet :).

Post reply on HN