11–20 of 141 posts
How does the esp-box compare? E.g. in a noisy environemnt, tv in the background, kids and dogs running around?
What are the biggest challenges that you see for improving it even further? Looks really promising!
Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…
Nice. Any bits of Mycroft in here? That project just imploded and I’m still sad about it.
A former Mycroft dev, Michael Hansen[1], is still building several year-of-the-voice projects after he was let go. I'm especially excited about Piper[2], which is a C++/py alternative to Mimic3.
[1] https://github.com/synesthesiam [2] https://github.com/rhasspy/piper
Nice. Any bits of Mycroft in here? That project just imploded and I’m still sad about it.
None.
The ESP BOX and ESP SR speech recognition library from Espressif handles the low-level audio stuff like wake word detection, DSP work for quality voice, voice activity detection, etc to get usable far-field audio. The wake word engine uses models from Espressif with wake words like "Alexa", "Hi ESP", "Hi Lexin", etc. If we get traction Espressif can make us a wake engine model for whatever we want (we're thinking "Hi Willow") but open to better ideas!
We currently stream audio after wake in realtime to our very high performance (optimized for "realtime" speech) Whisper inference server implementation. We plan to open source this next week.
We also patched in support for the most recent ESP SR version that has their actually amazingly good Multinet 6 speech command model that does recognition of up to 400 commands completely on device after wake activation. We currently try to pull light and switch entities from your configured Home Assistant instance to build the speech commands but it's really janky. We're working on this.
The default currently is to use our best-effort hosted inference server implementation but like I say in the README, etc we're open sourcing that next week so anyone can stand it up and do all of this completely locally/inside your walls.
Very interesting, I would buy an "off the shelf" version if it worked out of the box with Vicuna13 or similar LLM.
Our inference server also supports long-lived sessions via WebRTC for transcription, etc applications ;).
You can chain speech to text -> LLM -> text to speech completely in the inference server and input/output through Willow, along with other APIs or whatever you want.
I do not see anything posted on the Home Assistant (HA) Community forums.
> Configuring and building Willow for the ESP BOX is a multi-step process. We're working on improving that but for now...
This is crucial as your "competitors" are ready out of the box. I believe HA can be a Google/Alexa alternative to the masses only if the "out-of-the-box" experience is comparable to the commercial solutions.
Good luck, and keep us updated!
What are the biggest challenges that you see for improving it even further? Looks really promising!
Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…
That's really cool! Does this mean 400 specific commands, e.g. "turn on the living room lights" or 400 commands that can be applied to different targets, e.g. "turn on the X lights" where X is some light. (400 actually feels like it would be enough to speed up the vast majority of interactions either way, but I'm curious :)
What are the biggest challenges that you see for improving it even further? Looks really promising!
Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…
I'm curious if this is something lightweight enough that might be possible to run as a Home Assistant add-on on relatively low-powered hardware such as an RPi.
Sounds interesting - one question I have is about the mic array... Isn't this one of the supposed benefits of a physical Alexa device, and rumored to be sold at a loss because of the quality. How does the esp-box compare? E.g. in a noisy environemnt, tv in the background, kids and dogs running around?
Between that and the Espressif AFE (audio frontend interface) doing a bunch of DSP "stuff" in our testing it does remarkably well in noisy environments and far-field (25-30 feet) use cases.
Our inference server implementation (open source, releasing next week) uses a highly performance optimized Whisper which does famously well with less-than-ideal speech quality.
All in, even though it's all very early, it's very competitive with Echo, etc.
Earlier quoted context omitted.
Thanks! If I'm being perfectly honest I'm surprised we got it this far already. If I wanted to be really critical: - Far-field speech is actually kind of hard. There are at least dozens of "knobs" we can tweak between the various component libraries, etc to improve speech quality and reliability for more users in more environments. We've tested as much as we can considering there's only two of us but we need more tes…
> Open sourcing our inference server I'm curious if this is something lightweight enough that might be possible to run as a Home Assistant add-on on relatively low-powered hardware such as an RPi.
We'll also (likely) be creating our own TTS/STT HA component for our inference server that does some special/unique things to support Willow.
[0] - https://github.com/toverainc/willow/wiki/Home-Assistant