181–190 of 214 posts
It seems to fail. I sent the prompt: “ 5° warmer” And it said: “ setting the temperature to 5°F”
Either way, setting temperature to 5° is obviously wrong - even if it knew the current temperature - but models of this size can't reason about relative values very well.
Give it a tool to change temperature by a given amount, and see what happens!
Earlier quoted context omitted.
Tested your example, the confidence is 0. In smart home context, I can think of an application where the low confidence answers can be forwarded to cloud, whereas the vast majority generic queries solved locally, if the confidence is reliable enough. The response is quite fast by the way.
But this isn’t a query that should need to be forwarded to the cloud for acting on!
It cannot be used for direct customer interaction, it's simply too limited.
Nobody wants to say "lock the front door", they would just do it
And even quite direct commands like "tighten security" are not understood by the model
This is cool. I definitely think the "micro" sized LLM space is underappreciated, so it's always good to see work like this. I foresee a paradigm in some contexts where you have a hierarchy of LLMs, with more competent models actively training smaller models to solve specific tasks very efficiently, and something like this could be the smallest layer in that stack. With that being said, the web demo is not particular…
For one thing, on beefy-enough recent CPUs, you could keep the weights hot in the L2 cache of a single CPU core. (Which is clearly not the use-case, but might be interesting to those looking for extreme TPS numbers. Or perhaps for efficient training!)
I’d be interested in attempting to run this in a network- enabled 32MiB RAM microVM.
Then I tried using it on a raspberry pi 4. Did not work.
This is specifically for 64 bit arm. I see no reason why you could not enable desktops as well? Just a matter of compilation
Earlier quoted context omitted.
The wyoming protocoll seems to be the path for home assistant audio, so you need STT (wyoming-faster-whisper), TTS (wyoming-piper for wide language support) API endpoints and some Ollama or OpenAI API endpoint available for your home assistant server.
That is interesting, thanks. And I see separable wake-word detection there. I'd always assumed that was likely to be closer to the metal (more like recognise a sound than turn sound to word).
My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?
Try asking it to set the thermostat to a value. It's a very small model, there's not much reasoning capability
Otherwise why bother with an LM? 14MB can fit a lot of regexes for fixed sentences.