Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a fully local LLM voice assistant to control my smart home

johnthenerd.com

Post reply on HN
Print view View original

91–100 of 194 posts

Re: Building a fully local LLM voice assistant to control my smart home

#91

Post by nurettin »

rubymamis wrote: post #89
wokwokwok wrote: post #87

Was I the only who got to the end and was like, “and then…?” You installed it and customised your prompts and then… it worked? It didn’t work? You added the hugging face voice model? I appreciate the prompt, but broadly speaking it feels like there’s a fair bit of vague hand waving here: did it actually work? It mixtral good enough to consistently respond in an intelligent manner? My experience with this stuff has be…

I was expecting a video showing it in action...

I was expecting to see funny interactions between the user and their GlaDos prompt. And watching people respond to this post in serious LinkedIn tones is as hilarious as his project which seems to be tailored for a portal nerd.

Re: Building a fully local LLM voice assistant to control my smart home

#92

Post by jodrellblank »

jorvi wrote: post #33
polishdude20 wrote: post #30

Earlier quoted context omitted.

What about like, if I said "switch off the lamp at 3:45" How would you translate the Json you'd get out of that to get the same output? The subject would be "lamp" . Your app code would need to know that lamp is also light.

LLM just are waayyy too dangerous for something like home automation, until it becomes a lot more certain you can guarantee an output for an input. A very dumb innocuous example would be you ordering a single pizza for the two of you, then telling the assistant “actually we’ll treat ourselves, make that two”. Assistant corrects the order to two. Then the next time you order a pizza “because I had a bad day at work”,…

Chapter 4: In Which Phileas Fogg Astounds Passepartout, His Servant

Just as the train was whirling through Sydenham, Passepartout suddenly uttered a cry of despair.

"What's the matter?" asked Mr. Fogg.

"Alas! In my hurry—I—I forgot—"

"What?"

"To turn off the gas in my room!"

"Very well, young man," returned Mr. Fogg, coolly; "it will burn—at your expense."

- Around The World in 80 Days by Jules Verne, who knew that leaving the heat on while you went on vacation wouldn't burn down your house, 1872.

Re: Building a fully local LLM voice assistant to control my smart home

#93

Post by zer00eyz »

balloob wrote: post #52

Founder of Home Assistant here. Great write up! With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve: * I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be…

I just took break from messing with my HA install to read ... and low and behold!!!

First thanks for a great product, I'll be setting up a dev env in the coming weeks to fix some of the bugs (cause they are impacting me) so see you soon on that front.

As for the grammar and framework langchain might be what's your looking for on the LLM front. https://python.langchain.com/docs/get_started/introduction

Have you guys thought about the hardware barriers? Because most of my open source LLM work has been on high end desktops with lots of GPU, GPU ram and system ram? Is there any thought to Jetson as a AIO upgrade from the PI?

Re: Building a fully local LLM voice assistant to control my smart home

#94

Post by IshKebab »

balloob wrote: post #52

Founder of Home Assistant here. Great write up! With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve: * I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be…

Tell the LLM a Typescript API and ask it to generate a script to run in response to the query. Then execute it in a sandboxed JS VM. This works very well with ChatGPT. Haven't tried it with less capable LLMs.

Re: Building a fully local LLM voice assistant to control my smart home

#97

Post by JohnTheNerd »

wokwokwok wrote: post #87

Was I the only who got to the end and was like, “and then…?” You installed it and customised your prompts and then… it worked? It didn’t work? You added the hugging face voice model? I appreciate the prompt, but broadly speaking it feels like there’s a fair bit of vague hand waving here: did it actually work? It mixtral good enough to consistently respond in an intelligent manner? My experience with this stuff has be…

it actually works really well when I use it, but is slow because of the 4060Ti's (~8 seconds) and there is slight overfitting to the examples provided. none of it seemed to affect the actions taken, just the commentary.

I don't have prompts/a video demo on hand, but I might get and post them to the blog when I get a chance.

I didn't intend to make a tech demo, this is meant to help anyone else who might be trying to build something like this (and apparently HomeAssistant itself seems to be planning such a thing!).

Re: Building a fully local LLM voice assistant to control my smart home

#98

Post by vladgur »

"I expose HomeAssistant to the internet so I can use it remotely without a VPN,"

I wonder if this is a common use case? I would not want to expose Home Assistant to the internet because it requires trust in HASS that they keep an eye on vulnerabilities and trust in me that i update HASS regularly.

Do many Home assistant users do it? I prefer keeping it behind wireguard.

Re: Building a fully local LLM voice assistant to control my smart home

#99

Post by iamflimflam1 »

I played around doing a similar thing with the OpenAI APIs - it’s interesting to see how well it can interpret very vague requests.

https://youtu.be/BeJVv0pL5kY

You can really imagine how with more sensors feeding in the current state of things and having a history of past behaviour you could get some powerful results.

Re: Building a fully local LLM voice assistant to control my smart home

#100

Post by puchatek »

JohnTheNerd wrote: post #62
balloob wrote: post #52

Founder of Home Assistant here. Great write up! With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve: * I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be…

thank you for building an amazing product! I suspect cloning OpenAI's API is done for compatibility reasons. most AI-based software already support the GPT-4 API, and OpenAI's official client allows you to override the base URL very easily. a local LLM API is unlikely to be anywhere near as popular, greatly limiting the use cases of such a setup. a great example is what I did, which would be much more difficult witho…

So how much of the 8s is spent in the LLM vs Piper?

Some of the example responses are very long for the typical home automation usecase which would compound the problem. Ample room for GladOS to be sassy but at 8s just too tardy to be usable.

A different approach might be to use the LLM to produce a set of GladOS-like responses upfront and pick from them instead of always letting the LLM respond with something new. On top of that add a cache that will store .wav files after Piper synthesized them the first time. A cache is how e.g. Mycroft AI does it. Not sure how easy it will be to add on your setup though.

Post reply on HN