Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Willow – Open-source privacy-focused voice assistant hardware

github.com

Post reply on HN
Print view View original

71–80 of 141 posts

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#71

Post by COGlory »

kkielhofner wrote: post #67
RileyJames wrote: post #65

I’ve been living in a house for the past few months with a google assistant. I only use it to put on music, but I have noticed I play more music due to the ease of putting it on. But I hate the privacy invasion aspect. I’m definitely in the market for something like this. And this one looks great. Additionally, I’ve noticed that the google voice assistant (connected to Spotify) doesn’t keep playing the albums I ask f…

Music output is "on the list". Biggest fundamental issue is the speaker built in the ESP BOX is optimized for speech and not going to impress anyone playing music. That said, the ESP BOX (of course) supports bluetooth so we can definitely pair with a speaker you bring. Willow is the first of it's kind that I'm aware of to enable this kind of functionality at anything close to this price point in the open source ecosy…

Could we not use Willow to cast music, say, via Spotify or some other network streamer, through HA, to my pre-existing sound system?

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#72

Post by haarts »

kkielhofner wrote: post #7
tikkun wrote: post #2

What are the biggest challenges that you see for improving it even further? Looks really promising!

Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…

With regard to this:

> - On the wire/protocol stuff. We're doing pretty rudimentary "open new connection, stream voice, POST somewhere". This adds extra latency and CPU usage because of repeated TLS handshakes, etc. We have plans to use Websockets and what-not to cut down on this.

I've recently used the Noise protocol[1] to do some encrypted communication between two services I control but separated by the internet.

It was surprisingly easy!

[1]: https://noiseprotocol.org/

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#73

Post by macrolime »

Nice. Siri is completely unusable with an accuracy of less than 10%. I'm guessing Whisper on CPU is probably the same, so I wouldn't risk wasting time on trying the inference server if it only runs on CPU, but once that runs on GPU it would be cool to try this out.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#74

Post by kkielhofner »

_joel wrote: post #68

Is there an analogous thing for RPi? I've got some old ones and a USB mic array from seeed etc that I've still not put to use. Also got an ESP32 (vanilla with a small oled) if I can use that?

The Home Assistant project (as part of the "year of voice") is working on wake word, etc for Raspberry Pi from what I understand. However, as someone who's tried to do exactly this on a Raspberry Pi before supporting wake word and getting clean audio from 25 feet away with background noise, acoustic echo, etc with a random collection of software and hardware is very challenging. I have an entire graveyard of mic arrays from seeed and others myself :).

Espressif really did us all a solid with this hardware and their ADF and SR frameworks.

Whether it's cost, being fully assembled and ready to go, and even wake word, AEC, AGC, BSS, NS, etc at least as of now the ESP BOX is essentially impossible to compete with in terms of hardware in the open ecosystem.

I talk about this and more on our wiki pages[0] (check out "Hardware" and "Home Assistant"). In short, the Espressif frameworks we use /technically/ support the "regular" ESP32 but it's so limited (and the ESP BOX/ESP S3 is so cheap) we're not super interested in supporting it.

We're aiming for an end-user experience that's competitive with Echo, Google Home, etc in every possible way - speed, quality, reliability, functionality, and cost.

In fact, we want to crush them on all points to where there's no reason left to buy one of them.

[0] - https://github.com/toverainc/willow/wiki/

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#75

Post by yelite »

discardedrefuse wrote: post #48

So I was just looking at the installation process for this device's dev environment (ESP-IDF from espresiff) and it seems kind of...insane. The manual install method in the directions is not manual at all. It's a script that calls several python scripts. One has 2660 LOC and installs a root certificate (hard coded in the script itself) because of course, even though you just cloned the whole repo, it still has to dow…

If you are open to Nix, you can try https://github.com/mirrexagon/nixpkgs-esp-dev. I used it for a small project a while ago and the experience was pretty good.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#76

Post by Cheetah26 »

This looks like something I've been wanting to see for a while.

I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name.

I'd be great if this allows limiting / customizing what words and actions you want.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#77

Post by kkielhofner »

macrolime wrote: post #73

Nice. Siri is completely unusable with an accuracy of less than 10%. I'm guessing Whisper on CPU is probably the same, so I wouldn't risk wasting time on trying the inference server if it only runs on CPU, but once that runs on GPU it would be cool to try this out.

GPU (currently CUDA only) is our primary target for our inference server implementation. It "runs" on CPU but our goal is to enable an ecosystem that is competitive with Alexa in every possible way and even with the amazing work of whisper.cpp and other efforts it's just not happening (yet).

We're aware that's controversial and not really applicable to many home users - that's why we want to support any TTS/STT engine on any hardware supported by Home Assistant (or elsewhere) in addition to ESP BOX on device local command recognition.

But for the people such as yourself, and other commercial/power/whatever users our inference server that we're releasing next week that works with Willow provides impressive results - on anything from a GTX 1060 to an H100 (we've tested and optimized for anything in between the two).

We use ctranslate2 (like faster-whisper) and some other optimizations for performance improvements and conservative VRAM usage. We can simultaneously load large-v2, medium, and base on a GTX 1060 3GB and handle requests without issue.

Again, it's controversial but the fact remains a $100 Tesla P4 that idles at 5 watts and has max TDP of 60 watts from eBay with our inference server implementation does the following:

large-v2, beam 5 - 3.8s of speech, inference time 1.1s

medium, beam 1 (suitable for Willow tasks) - 3.8s of speech, inference time 588ms

medium, beam 1 (suitable for Willow tasks), 29.2s of speech, inference time 1.6s

An RTX 4090 with large-v2, beam 5 does 3.8s of speech in 140ms and 29.2s of speech with medium beam 1 (greedy) in 84ms.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#78

Post by chankstein38 »

Cheetah26 wrote: post #76

This looks like something I've been wanting to see for a while. I currently have a google home and I'm getting increasingly fed up with it. Besides the privacy concerns, it seems like it's getting worse at being an assistant. I'll want my light turned on by saying "light 100" (for light to 100 percent) and it works about 80% of the time, but the others it starts playing a song with a similar name. I'd be great if thi…

This drives me nuts and happens all the time as well. To be honest, I unplugged my google home device a while back and haven't missed it. It mostly ended up being a clock for me because I'd try to change the color of my lights to a color that it mustn't have been capable of because I'd have to sit there for minutes listening to it list stores in the area that might sell those colored lights or something. It wouldn't stop. This is just one of many frustrating experiences I'd had with that thing.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#79

Post by CharlesW »

macrolime wrote: post #73

Nice. Siri is completely unusable with an accuracy of less than 10%. I'm guessing Whisper on CPU is probably the same, so I wouldn't risk wasting time on trying the inference server if it only runs on CPU, but once that runs on GPU it would be cool to try this out.

> Siri is completely unusable with an accuracy of less than 10%.

That seems unusual. I've been using both for the last few weeks while replacing my Homebridge setup, and Siri has been as accurate as Alexa — good enough that I've decided that I can now leave the Alexa ecosystem. To be more specific, both are (conservatively) 95%+ accurate for my home control scenarios.

Re: Show HN: Willow – Open-source privacy-focused voice assistant hardware

#80

Post by chankstein38 »

RileyJames wrote: post #65

I’ve been living in a house for the past few months with a google assistant. I only use it to put on music, but I have noticed I play more music due to the ease of putting it on. But I hate the privacy invasion aspect. I’m definitely in the market for something like this. And this one looks great. Additionally, I’ve noticed that the google voice assistant (connected to Spotify) doesn’t keep playing the albums I ask f…

It also, at least in my case, frequently won't stop playing when you tell it to. And, if you want a song that has a title that isn't family friendly, it'll completely ignore that title and just play whatever the heck it wants.
Post reply on HN