[flagged]
Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
111–120 of 188 posts
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#112How many languages does it supports in such a small size?
Tested with german and this kinda works, but confidence suffers.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#113[flagged]
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#114My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?
Tested your example, the confidence is 0. In smart home context, I can think of an application where the low confidence answers can be forwarded to cloud, whereas the vast majority generic queries solved locally, if the confidence is reliable enough. The response is quite fast by the way.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#115Nice. I used Needle as a router in https://rcarmo.github.io/projects/memento/, need to take a look at this
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#116Curious, why did you go down to 2bit rather than 4 bits? 4bit with folding the layers should arrive at the same size, but with better quality?
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#117It's definitely cool that you can get any reasoning whatsoever out of such a small model. That said, its reasoning is "interesting":
Query: "Make the living room dark" Agent: "User wants lights on in living room. 'dark' implies dim. Room 'living room', action 'on'." (And on every test I did, it just completely ignored the "brightness" parameter)
It also appears to have no concept of what a door or light actually is, whenever the query diverges from "Lock door X" or "Turn on light X", it tries to shoehorn whatever additional context is given into the device name:
Query: "Lock out the vacuum salesman at the front door" Agent tries to lock "front door vacuum salesman"
"The way you talk really makes me appreciate silence" is classified as "positive" with 82% confidence.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#118Earlier quoted context omitted.
The best entrypoint is Home Assistant: https://www.home-assistant.io/ That will get you a lot further than what you're asking, but if you dig a bit through Home Assistant features, resources, etc., you may find the current "best" answers to your questions. If you want a quick answer: Whisper is a good open-source speech-to-text model which comes in a variety of sizes ( https://huggingface.co/openai/whisper-tiny ). Yo…
The wyoming protocoll seems to be the path for home assistant audio, so you need STT (wyoming-faster-whisper), TTS (wyoming-piper for wide language support) API endpoints and some Ollama or OpenAI API endpoint available for your home assistant server.
That is interesting, thanks. And I see separable wake-word detection there. I'd always assumed that was likely to be closer to the metal (more like recognise a sound than turn sound to word).
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#119It's definitely cool that you can get any reasoning whatsoever out of such a small model. That said, its reasoning is "interesting": Query: "Make the living room dark" Agent: "User wants lights on in living room. 'dark' implies dim. Room 'living room', action 'on'." (And on every test I did, it just completely ignored the "brightness" parameter) It also appears to have no concept of what a door or light actually is,…
that model is 14MB large what do you expect. but I agree it's funny regardless
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#120Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…
It seems to love locking doors. I was hoping this would dim the lights.
> I'm hungover
{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. 'hungover' implies a security door. No specific door named, so use 'front door' as default.", "confidence": 0 }