51–60 of 194 posts
Earlier quoted context omitted.
What about like, if I said "switch off the lamp at 3:45" How would you translate the Json you'd get out of that to get the same output? The subject would be "lamp" . Your app code would need to know that lamp is also light.
In all seriousness, I have names for my lights for this very reason.
With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve:
* I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be able to query that model to see what the model is capable off.
* I want to see local LLMs with support for a feature similar or equivalent to OpenAI functions. We cannot include all possible information in the prompt and we need to allow LLMs to make actions to be useful. Constrained grammars do look like an possible alternative. Creating a prompt to write JSON is possible but need quite an elaborate prompt and even then the LLM can make errors. We want to make sure that all JSON coming out of the model is directly actionable without having to ask the LLM what they might have meant for a specific value.
Earlier quoted context omitted.
So you'd need to somehow know that a lamp is also a light eh
With a proper grammar, you can require the "subject" field to be one of several valid entity names. In the prompt, you would tell the LLM what the valid entity names are, which room each entity is in, and a brief description of each entity. Then it would be able to infer which entity you meant if there is one that reasonably matches your request. If you're speaking through the kitchen microphone (which should be prov…
I also ended up writing a classifier using some python library that seems to outperform home assistant's implementation. Not sure what the issue is there. I just followed the instructions from an LLM and the internet.
I want my new assistant to be sassy and sarcastic.If Mixtral doesn't support system prompts, and you just copy in your system prompts as another "user" message, does that suggest that Mixtral is less resilient to prompt injection than commercial models, because it doesn't have any concept of "trust this instruction more than this other class of instruction"?
It’s also why it is so good, I have some document summarization tasks that includes porn sites and other LLM refuse to do it. Mixtral doesn’t care.
Founder of Home Assistant here. Great write up! With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve: * I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be…
Currently pushing for application note https://github.com/Mozilla-Ocho/llamafile/pull/178 to encourage integration. Would be good to hear your thoughts on making it easier for home assistant to integrate with llamafiles.
Also as an idea, maybe you could certify recommendations for LLM models for home assistant. Maybe for those specifically trained to operate home assistant you could call it "House Trained"? :)
Founder of Home Assistant here. Great write up! With Home Assistant we plan to integrate similar functionality this year out of the box. OP touches upon some good points that we have also ran into and I would love the local LLM community to solve: * I would love to see a standardized API for local LLMs that is not just a 1:1 copying the ChatGPT API. For example, as Home Assistant talks to a random model, we should be…
I find the whole area fascinating. I’ve spent an unhealthy amount of time improving “Siri” by using some of the work from the COPILOT iOS Shortcut and giving it “functions” which are really just more iOS Shortcuts to do things on the phone like interact with my calendar. I’m using GPT-4 but it would be amazing to break free of OpenAI since they’re not so open and all.