Mycroft – open source voice assistant

It's not dying at all - it's an incredibly useful interaction style.

Those companies are failing to profit because they don't understand that a digital assistant needs to be working with me, locally, and not subverting my intent.

It just needs to be my device, and not a sales rep for google/amazon. I use voice controls all the time at home - it's astoundingly useful in all sorts of situations, and I'm not even disabled (where it's literally life changing in some cases).

I don't think the entire space is dying. Amazon is having problems because Alexa has little benefit to them outside of direct monetization. They wanted people to use Alexa to buy things and no one wants to shop like that. So they have these devices and all this infrastructure to run people's kitchen timers, lights and play music. People will buy Alexa devices on Prime Day, use them dozens of times a day for years, and never make a dime for Amazon.

Apple isn't necessarily in the same boat. Siri isn't particularly good, but it does all those things well. Most importantly, it keeps people on iPhones and in the Apple ecosystem, which does make money.

You already have the phone and probably have a device that works with HomeKit so why not try it out. Next, you buy some new lights. Before you know it, you're controlling most of the lights in your house, streaming Apple Music and setting kitchen timers from your Apple Watch. Next time you need a new phone, you're not even going to think about anything else because if you change you won't be able to turn on your lights anymore.

Apple has a plan that works and Amazon doesn't.

Mark II sounds like he'd rather be sleeping than helping the person in the demo video.

My needs for a smart speaker are not really passing a turing test. I want them to be automations. They need to get some things that an AI would do right, but there is a large step between an assistant that can do specific things and an AI that can talk about anything.

I played with Mycroft about two years ago. I had been using a couple Google home minis for a while for the usual things (play spotify, set timers, ask the weather, control lights around the hose). They worked perfectly for that. At the time I decided to de-Google my life and take back my privacy so I went looking for something open source that would provide me more control of my data. I found Mycroft and played with it for a few months.

I was pretty excited about it. I bought a ReSpeaker 2.0, which is an embedded device that can run Linux and has a six microphone array. I designed a custom 3d-printed case to hold the ReSpeaker and a small speaker to make my own little "Jarvis" box (Iron-man reference).

My favorite part about the whole thing was the customization. I wrote a couple of skills to do some other things for me. For example, I could say "Where can I watch X?" and it would use an API to search for a TV show or movie to see where it was available on Netflix, Amazon Prime, Disney+, etc and let me know. It's always been annoying to go Google and try to figure out where I can watch something streaming online, but limited to only the services I currently subscribe to. I wrote another skill that tied into my couchpotato instance so I could say "Download the movie X" and it would go find it and download it. If it found multiple matches, it would read off the top few matches and let me choose the correct one. I even tied those skills together so if the first skill couldn't find a movie at one of my streaming services it would ask if I wanted to download it and I could simply say "yes". I also modified the code to use a custom text to speech API so I could configure Mycroft to use a custom voice.

It was all really cool and I had a lot of fun playing with it. The biggest problem I ran into was the wake word recognition. It worked mostly OK for me on the ReSpeaker from close range but I found as I moved away it went downhill. It was especially bad if I had my device playing music, which is possibly the most common thing I was using my Google Home mini for. I had hoped that the ReSpeaker would help with this, because it had the six microphone array and some built-in loopback hardware to try and cancel out any noise that that was being generated by the ReSpeaker. So any sound output to the speakers would be looped back into the ReSpeaker and could be subtracted from the microphone's input. I found that I just couldn't get it to work well, though. I think the music was causing vibrations that were overloading the microphone array and causing it to be unable to hear me through the music. It's possible it could be improved with a better hardware design to help reduce vibration caused by the device's own speaker. Maybe it works better now, two years later. I think I had configured Mycroft to use Snowboy for wake-word recognition so I could name my Mycroft something else (Jarvis).

One day the Mycroft installation just stopped working on my device after I hadn't touched it in a week or more and I never went back to figure out what was wrong. It's still sitting on the corner of my desk unplugged. If I could have got the wake-word recognition working reliably with music playing I think I would have used it a lot, but I wasn't able to at the time.

I just recently bought a smart watch with a built in "Alexa" app that allows you to send voice commands to your phone which get processed through the watch's official app. I'm instead using Gadgetbridge on Android to interface to the watch. Some kind hacker updated Gadgetbridge to add very basic support for my watch's microphone, allowing you to send the raw voice data to an external application. I'm hoping I'll be able to use this to revive my Mycroft instance and I'll just send voice commands to Mycroft from my watch/phone via a custom Android app/service. In theory, I'll be wearing the watch all the time anyway and having the microphone on my person and right next to my face should hopefully help with the speech-to-text and I won't have to worry about a wake word at all. I've only just barely started working on this, though.