What is the difference between this and random sentence generator?
Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
11–20 of 175 posts
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#12Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…
The website says the model is for "tool calling, device use, and structured extraction". Your example just doesn't seem to be very relevant. FWIW, it did a pretty good job for tool calling when I tried it, and I think it could be pretty nice to have this running on locally and integrate with Home Assistant.
False positives are definitely relevant and worth measuring - natural language interfaces always have a discoverability problem, i.e., users not knowing what actions the system does and does not support. If the frontend of that system lacks the ability to reject unsupported commands, weird stuff happens.
Nonetheless, this is very cool work! If I can offer a small suggestion to the team at Cactus, it would be to evaluate your releases on some usability criteria (including false positives). Any serious integrator or adopter of these models would want to have that information available.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#13Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…
"confidence": 0, so I guess you could threshold it
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#14That's really cool - I was already thinking of compressing `functiongemma-270m-it` down to 1-2 bits so it would work flawlessly in the browser. Your `Fine-tuning` feature is even much more convenient.
Thanks, give it a splin!
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#15Was really cool to see yous use Engrams to cut down compute!
Given its basically an O(1) lookup with disk space being the main constraint, I was curious if you've tried ablating engram layers and sizes across your setup?
Also, why mHC over attention residuals?
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#16Naïve and clumsy question: how would you pair this with speech-text-speech stuff, wake words etc.? Are there good examples of this for a Pi 5? The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device. ETA: pun not intended
Users often stack a transcription model on top to get the voice prompt, then decode to actions. Think of Alexa and Siri.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#17Earlier quoted context omitted.
The website says the model is for "tool calling, device use, and structured extraction". Your example just doesn't seem to be very relevant. FWIW, it did a pretty good job for tool calling when I tried it, and I think it could be pretty nice to have this running on locally and integrate with Home Assistant.
False positives are definitely relevant and worth measuring - natural language interfaces always have a discoverability problem, i.e., users not knowing what actions the system does and does not support. If the frontend of that system lacks the ability to reject unsupported commands, weird stuff happens. Nonetheless, this is very cool work! If I can offer a small suggestion to the team at Cactus, it would be to evalu…
> "confidence": 0
OP and the linked page talk about the confidence score and using it as an action threshold, so it looks like an appropriate total response to me.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#18What is the difference between this and random sentence generator?
Random sentence is not a function call.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#19Naïve and clumsy question: how would you pair this with speech-text-speech stuff, wake words etc.? Are there good examples of this for a Pi 5? The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device. ETA: pun not intended
The best entrypoint is Home Assistant: https://www.home-assistant.io/
That will get you a lot further than what you're asking, but if you dig a bit through Home Assistant features, resources, etc., you may find the current "best" answers to your questions.
If you want a quick answer: Whisper is a good open-source speech-to-text model which comes in a variety of sizes (https://huggingface.co/openai/whisper-tiny). You can definitely get something like this running on a Pi 5. There are plenty of other STT models out there, some of which are built specifically for this context (again, see the Home Assistant stuff), but Whisper comes up a lot as a good default choice.
So with something like Whisper, you could just have a simple script which is constantly listening to a rolling window of audio and transcribing it. When the transcription includes a key phrase, you can pass the rest of the transcription to Needle2 (or anything else for that matter). From there, you take the results and execute the necessary tool calls.
There's a bit more to all of this to make it work smoothly, but fundamentally this is all there is to it. All this would work very fast on a Pi 5 (although I wouldn't expect the results to be particularly good without some serious hand-crafted logic, fine-tuning, etc.). If you want to mess around this stuff, handing all of this to Claude, Codex, etc., can get you something spun up and functional very quickly.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#20Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…
yeah I got the same, almost like its biased heavily towards that as the 0 ranking -- my prompt was just the word 'potato'