Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Willow – Open-source privacy-focused voice assistant hardware

github.com

Post reply on HN
Print view View original

81–90 of 141 posts

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#81

Post by kkielhofner »

yelite wrote: post #75
discardedrefuse wrote: post #48

So I was just looking at the installation process for this device's dev environment (ESP-IDF from espresiff) and it seems kind of...insane. The manual install method in the directions is not manual at all. It's a script that calls several python scripts. One has 2660 LOC and installs a root certificate (hard coded in the script itself) because of course, even though you just cloned the whole repo, it still has to dow…

If you are open to Nix, you can try https://github.com/mirrexagon/nixpkgs-esp-dev . I used it for a small project a while ago and the experience was pretty good.

Nice!

For anyone who would try to use this with Willow (I like the effort and CERTAINLY don't love the ESP dev environment as-is):

- ESP ADF is actually the root of the project. ESP-IDF and all other sub-components are components themselves to ADF.

- We use bleeding edge ESP SR[0] that we also include as an ADF component.

- Plus LVGL, ESP-DSP, esp-lcd-touch, and likely others I'm forgetting ATM.

[0] - https://github.com/espressif/esp-sr

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#82

Post by kkielhofner »

Cheetah26 wrote: post #76

This looks like something I've been wanting to see for a while. I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name. I'd be great if thi…

Totally get it!

There are at least two ways to deal with this frustrating issue with Willow:

- With local command recognition via ESP SR command recognition runs completely on the device and the accepted command syntax is defined. It essentially does "fuzzy" matching to address your light command ("light 100") but there's no way it's going to send some random match to play music.

- When using the inference server -or- local recognition we send the speech to text output to the Home Assistant conversation/intents[0] API and you can define valid actions/matches there.

[0] - https://developers.home-assistant.io/docs/intent_index/

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#83

Post by kkielhofner »

COGlory wrote: post #71
kkielhofner wrote: post #67

Earlier quoted context omitted.

Music output is "on the list". Biggest fundamental issue is the speaker built in the ESP BOX is optimized for speech and not going to impress anyone playing music. That said, the ESP BOX (of course) supports bluetooth so we can definitely pair with a speaker you bring. Willow is the first of it's kind that I'm aware of to enable this kind of functionality at anything close to this price point in the open source ecosy…

Could we not use Willow to cast music, say, via Spotify or some other network streamer, through HA, to my pre-existing sound system?

The approach there would be to ignore Willow for music output and just do what it does today:

- Wake

- Get command

- Send to Home Assistant conversation/intents API[0]

- Home Assistant does whatever you define, including what you describe just like it does today

So unless I'm missing something your use case should "just work".

[0] - https://developers.home-assistant.io/docs/intent_index/

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#84

Post by macrolime »

CharlesW wrote: post #79
macrolime wrote: post #73

Nice. Siri is completely unusable with an accuracy of less than 10%. I'm guessing Whisper on CPU is probably the same, so I wouldn't risk wasting time on trying the inference server if it only runs on CPU, but once that runs on GPU it would be cool to try this out.

> Siri is completely unusable with an accuracy of less than 10%. That seems unusual. I've been using both for the last few weeks while replacing my Homebridge setup, and Siri has been as accurate as Alexa — good enough that I've decided that I can now leave the Alexa ecosystem. To be more specific, both are (conservatively) 95%+ accurate for my home control scenarios.

I've never tried any voice recognition system that works well. Maybe my accent is too different from typical training data or something. I had a voice recognition program on my computer in 1994 that had about the same accuracy for me as any modern voice recognition system that I have tried.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#85

Post by schainks »

Cheetah26 wrote: post #76

This looks like something I've been wanting to see for a while. I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name. I'd be great if thi…

THIS. It's hilarious and infuriating our digital assistants struggle to understand variants of "set lights at X% intensity".

However, if I spend the time to configure a "scene" with the right presets, Google has no issue figuring it out.

If only it could notice regular patterns about light settings and offer suggestions that I could approve/deny.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#86

Post by t-vi »

Cheetah26 wrote: post #76

This looks like something I've been wanting to see for a while. I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name. I'd be great if thi…

Personally, I plugged a Jabra conference speaker to a Raspberry and if it hears something interesting, it sends to my local GPU computer for decoding (with whisper) + answer-getting + response sent back to the Raspberry as audio (with a model from coqui-ai/TTS but using more plain PyTorch). Works really nicely for having very local weather, calendar, ...

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#87

Post by _joel »

kkielhofner wrote: post #74
_joel wrote: post #68

Is there an analogous thing for RPi? I've got some old ones and a USB mic array from seeed etc that I've still not put to use. Also got an ESP32 (vanilla with a small oled) if I can use that?

The Home Assistant project (as part of the "year of voice") is working on wake word, etc for Raspberry Pi from what I understand. However, as someone who's tried to do exactly this on a Raspberry Pi before supporting wake word and getting clean audio from 25 feet away with background noise, acoustic echo, etc with a random collection of software and hardware is very challenging. I have an entire graveyard of mic arra…

Thanks, yes that's perfectly reasonable. Cheers for the reply.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#88

Post by kkielhofner »

haarts wrote: post #72
kkielhofner wrote: post #7

Earlier quoted context omitted.

Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…

With regard to this: > - On the wire/protocol stuff. We're doing pretty rudimentary "open new connection, stream voice, POST somewhere". This adds extra latency and CPU usage because of repeated TLS handshakes, etc. We have plans to use Websockets and what-not to cut down on this. I've recently used the Noise protocol[1] to do some encrypted communication between two services I control but separated by the internet.…

Thanks for mentioning noise! I've certainly looked at it before but our challenge is the sheer scope of what we're doing. Not to mention (similar to WebRTC that people have asked about) I'm not completely understanding the fit and benefit for our use case and application.

I talk about websockets because they achieve our mission and goal (in this case shaving milliseconds off command -> action -> confirmation) with robust, battle-tested client implementations already available in the ESP framework libraries. Same thing for MQTT. Both are supported by Home Assistant (and almost everything else in the space) today.

Because of this existing framework support, we'll have websockets done today-ish. Then we can (for now) move on to all of the other things people have asked for :). Hah, priorities!

Not saying Noise won't/can't ever happen - just that this is a very ambitious project as it stands and we have plenty of work to do all over the place :)!

Want to write a noise implementation for ESP IDF :)?

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#89

Post by macrolime »

kkielhofner wrote: post #77
macrolime wrote: post #73

Nice. Siri is completely unusable with an accuracy of less than 10%. I'm guessing Whisper on CPU is probably the same, so I wouldn't risk wasting time on trying the inference server if it only runs on CPU, but once that runs on GPU it would be cool to try this out.

GPU (currently CUDA only) is our primary target for our inference server implementation. It "runs" on CPU but our goal is to enable an ecosystem that is competitive with Alexa in every possible way and even with the amazing work of whisper.cpp and other efforts it's just not happening (yet). We're aware that's controversial and not really applicable to many home users - that's why we want to support any TTS/STT engin…

You've convinced me. Just ordered an ESP-BOX :p

Got a Home Assistant Yellow not long ago, so would be nice to get some decent voice control for it.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#90

Post by kkielhofner »

t-vi wrote: post #86
Cheetah26 wrote: post #76

This looks like something I've been wanting to see for a while. I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name. I'd be great if thi…

Personally, I plugged a Jabra conference speaker to a Raspberry and if it hears something interesting, it sends to my local GPU computer for decoding (with whisper) + answer-getting + response sent back to the Raspberry as audio (with a model from coqui-ai/TTS but using more plain PyTorch). Works really nicely for having very local weather, calendar, ...

Neat!

If you don't mind my asking, what do you mean "if it hears something interesting"? Is that based on wake word, or always listen/process?

Post reply on HN