Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a fully local LLM voice assistant to control my smart home

johnthenerd.com

Post reply on HN
Print view View original

41–50 of 194 posts

Re: Building a fully local LLM voice assistant to control my smart home

#41

Post by JohnTheNerd »

Havoc wrote: post #37

Why 4060s? I’d have gone for 2nd hand 3090s personally

power consumption. I am running multiple GPUs somewhere residential. the 4060Ti only draws 180W at max load (which it almost never reaches). 3090 is about double for 1.5x the VRAM, and it's notorious for briefly consuming much more than its rated wattage.

this isn't just about the power bill. consider that your power supply and electrical wiring can only push so many watts. you really don't want to try to draw more than that. after some calculations given my unique constrains, I decided 4060Ti is the much safer choice.

Re: Building a fully local LLM voice assistant to control my smart home

#42

Post by amluto »

jorvi wrote: post #39
coder543 wrote: post #35

Earlier quoted context omitted.

No. LLMs do not have memory like that (yet). Your 'scary' examples are very hypothetical and would require intentional design to achieve today; they would not happen by accident.

I love how burning your house down is something that deserves air quotes according to you. All I can tell you is this: LLM’s frequently misinterpret, hallucinate and “lie”. Good luck.

Preventing burning your house down belongs on the output handling side, not the instruction processing side. If there is any output from an LLM at all that will burn your house down, you already messed up.

Re: Building a fully local LLM voice assistant to control my smart home

#43

Post by polishdude20 »

canada_dry wrote: post #36
polishdude20 wrote: post #30

Earlier quoted context omitted.

What about like, if I said "switch off the lamp at 3:45" How would you translate the Json you'd get out of that to get the same output? The subject would be "lamp" . Your app code would need to know that lamp is also light.

User: in the sentence "switch off the lamp at 3:45" output the subject, action, time, and location as json Llama: { "subject": "lamp", "action": "switch off", "time": "3:45", "location": "" } Where there is an empty parameter the code will try to look back to the last recent commands for context (e.g. I may have just said "turn on the living room light"). If there's an issue it just asks for the missing info. Transla…

So you'd need to somehow know that a lamp is also a light eh

Re: Building a fully local LLM voice assistant to control my smart home

#44

Post by coder543 »

polishdude20 wrote: post #43
canada_dry wrote: post #36

Earlier quoted context omitted.

User: in the sentence "switch off the lamp at 3:45" output the subject, action, time, and location as json Llama: { "subject": "lamp", "action": "switch off", "time": "3:45", "location": "" } Where there is an empty parameter the code will try to look back to the last recent commands for context (e.g. I may have just said "turn on the living room light"). If there's an issue it just asks for the missing info. Transla…

So you'd need to somehow know that a lamp is also a light eh

With a proper grammar, you can require the "subject" field to be one of several valid entity names. In the prompt, you would tell the LLM what the valid entity names are, which room each entity is in, and a brief description of each entity. Then it would be able to infer which entity you meant if there is one that reasonably matches your request.

If you're speaking through the kitchen microphone (which should be provided as context in the LLM prompt as well) and there are no controllable lights in that room, you could leave room in the grammar for the LLM to respond with a clarifying question or an error, so it isn't forced to choose an entity at random.

Re: Building a fully local LLM voice assistant to control my smart home

#45

Post by dr_dshiv »

canada_dry wrote: post #13

I've been testing various LLMs (that can run locally - sans cloud) and (for example) the llava-v1.5-7b-q4 does a decent job for home automation. Example: I give the LLM a range of 'verbal' instructions related to home automation to see how well they can identify the action, timing, and subject: User: in the sentence "in 15 minutes turn off the living room light" output the subject, action, time, and location as json…

> Several of the latest models are on par to the results from Gpt4 in my tests.

Wow! So almost as good as alexa?

Re: Building a fully local LLM voice assistant to control my smart home

#46

Post by canada_dry »

wslh wrote: post #12
JohnTheNerd wrote: post #9

Earlier quoted context omitted.

that is correct! I would much rather run everything in-house, where I know the quality won't be degraded over time (see the Google Assistant announcement from yesterday) and I am in full control of my data. using a cloud service is much easier and cheaper, but I was not comfortable with that trade-off.

Based on your experience and existing code, it is easy to add continuous listening? Have not tested it but probably is already there. For example, I would like to have it always turned on and speaking to it about ideas at random times.

The solution I've got (in alpha) is a basic webcam that detects when you're looking at it.

The cam is positioned higher than most things in the room to reduce triggering it unnecessarily.

When it triggers (currently using just simple cvv facial landmark detection) it emits a beep and then listens for a verbal command.

Re: Building a fully local LLM voice assistant to control my smart home

#47

Post by AdrienBrault »

dr_dshiv wrote: post #45
canada_dry wrote: post #13

I've been testing various LLMs (that can run locally - sans cloud) and (for example) the llava-v1.5-7b-q4 does a decent job for home automation. Example: I give the LLM a range of 'verbal' instructions related to home automation to see how well they can identify the action, timing, and subject: User: in the sentence "in 15 minutes turn off the living room light" output the subject, action, time, and location as json…

> Several of the latest models are on par to the results from Gpt4 in my tests. Wow! So almost as good as alexa?

Probably much better than alexa. Gpt 3.5 is miles ahead alexa

Re: Building a fully local LLM voice assistant to control my smart home

#48

Post by lacrimacida »

jorvi wrote: post #39
coder543 wrote: post #35

Earlier quoted context omitted.

No. LLMs do not have memory like that (yet). Your 'scary' examples are very hypothetical and would require intentional design to achieve today; they would not happen by accident.

I love how burning your house down is something that deserves air quotes according to you. All I can tell you is this: LLM’s frequently misinterpret, hallucinate and “lie”. Good luck.

Im not taken aback by the current AI hype but having LLMs as an interface to voice commands is really revolutionary and a good fit to this problem. It’s just an interface to your API that provides the function as you see fit. And you can program it in natural language.

Re: Building a fully local LLM voice assistant to control my smart home

#49

Post by ilaksh »

jasonjmcghee wrote: post #28
canada_dry wrote: post #13

I've been testing various LLMs (that can run locally - sans cloud) and (for example) the llava-v1.5-7b-q4 does a decent job for home automation. Example: I give the LLM a range of 'verbal' instructions related to home automation to see how well they can identify the action, timing, and subject: User: in the sentence "in 15 minutes turn off the living room light" output the subject, action, time, and location as json…

Out of curiosity what are you using the vision aspect for? Fwiw bakllava is a much more recent model, using mistral instead of llama. Same size and capabilities

Does anyone know if there is something like bakllava but with commercial use permitted?

Re: Building a fully local LLM voice assistant to control my smart home

#50

Post by gerdesj »

Lienetic wrote: post #29
gerdesj wrote: post #24

Thank you so much for this write up mate. I'm fine with the usual systems n networking stuff but the AI bits and bobs is a bit of a blur to me, so having a template to start off with is a bit of a God's send. I'm a bit of a Home Assistant fan boi. I have eight of them to look after now. They are so useful as a "box that does stuff" on customer sites. I generally deploy HA Supervised to get a full Linux box underneath…

Can you share a bit more about why you're deploying HA in customer sites? I'm also a fan of HA and am interested to learn more about what you're doing and how it's going!

Here's how shit happens! We move to remote working due to a pandemic. Many of my customers do CAD on powerful gear in the office. They also have a ISO14001 registration (environmental standard) or not but want these gas guzzlers shut down at night.

So they want to be able to wake up their PCs and shut them down remotely. I'm already flooded with VPN requirements and the other day to day stuff. I recall an add on for HA for a Windows remote shutdown and I know HA can do "wake on LAN". ... and HA has an app.

I won't deny it is a bit of a fiddle, thanks to MS's pissing around with power management etc. When a Windows PC is shutdown, it isn't really and will generally only honour the BIOS settings once. You have to disable Windows's network card power management and it doesn't help that the registry key referring to the only NIC is sometimes not the obvious one.

Home Assistant has "HACS" for adding even more stuff and one handy addition is a restriction card - https://community.home-assistant.io/t/lovelace-restriction-c...

Anyway, the customer has the app on their phone. They have a dashboard with a list of PCs. Those cards are "locked" via restriction card. You have to unlock the card for your PC which has a switch to turn it on and off. The unlock thing is to avoid inadvertent start ups/down.

That is just one use - two customers so far use that. We also see "I've got a smart ... thing, can you watch it? ... Yes!

Zwave and Zigbee dongles cost very little and coupled with a laptop with probably bluetooth built in and HA, you get a lot of "can I ..."

Post reply on HN