these micro LLMs could be a game changer for hearing aids so many interesting lowfi hardware use cases
Exactly the use cases we designed Needle for!
Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
131–140 of 188 posts
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#132Any plan to release on ollama?
Yep
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#133Utterly Fascinating! For the longest time, I conceptualized LLM's as Text Input -> Text Output transformers, then later as Text Input -> Video Output transformers. Later still I conceptualized them (if they were general purpose) as Any Format Input -> Any Format Output transformers... The idea of a smaller parameter model runable on smaller/slower/less complex hardware (computers with no GPU, slower CPU's, less memor…
You explained it better, we've done a bad job at communicating its nice :(
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#134"make it as dark as possible" { "function_calls": [ { "name": "set_thermostat", "arguments": { "temperature": 72, "mode": "cool", "room": "living room" } } ], "reasoning": "'as dark as possible' -> set_thermostat to warm; 'dark' implies higher temperature; 'cool' mode for darkness.", "confidence": 0 } ... maybe this counts as dark humor at least. Since it seems limited to matching a few templates and otherwise fallin…
SO when confused, it gives you a low confidence score as a signal, in which case its recommended to finetune using th python package, train on your mac/PC
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#135This is very interesting! I'm going to spend some time with this. This is really the only class of LLM I'm interested in at all. I sincerely hope on-device takes over and everyone looses their asses on these data centers.
Thanks! Let us know how it goes :)
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#136"make it as dark as possible" { "function_calls": [ { "name": "set_thermostat", "arguments": { "temperature": 72, "mode": "cool", "room": "living room" } } ], "reasoning": "'as dark as possible' -> set_thermostat to warm; 'dark' implies higher temperature; 'cool' mode for darkness.", "confidence": 0 } ... maybe this counts as dark humor at least. Since it seems limited to matching a few templates and otherwise fallin…
I also wonder how small a LLM trained on catching only subject (e.g. living room) and action (light on) from text input could be compared to needle - the json wrapping could be done afterwards using templates.
The problem is the target device, an LLM can't run on an average TV well.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#137Could someone please share how such open source micro-LLMs might have been created? Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?
Another option for something this small and narrowly specialized could be to get traditional LLMs to synthesize the training data. Model collapse is probably less of an issue at this size relative to terabyte sized models.
Your thinking is correct haha
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#138Nice. I used Needle as a router in https://rcarmo.github.io/projects/memento/ , need to take a look at this
Insane!
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#139It's definitely cool that you can get any reasoning whatsoever out of such a small model. That said, its reasoning is "interesting": Query: "Make the living room dark" Agent: "User wants lights on in living room. 'dark' implies dim. Room 'living room', action 'on'." (And on every test I did, it just completely ignored the "brightness" parameter) It also appears to have no concept of what a door or light actually is,…
Ok, this is genuinely funny, we will fix these as we iterate, thanks lol.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#140This is cool. I definitely think the "micro" sized LLM space is underappreciated, so it's always good to see work like this. I foresee a paradigm in some contexts where you have a hierarchy of LLMs, with more competent models actively training smaller models to solve specific tasks very efficiently, and something like this could be the smallest layer in that stack. With that being said, the web demo is not particular…
I'm quite impressed by the results of the web demo, especially given its size and the precision with which it uses the three available tools (tested with German commands). I could imagine that this LLM would fit well into a setup with multiple micro-sized LLMs for different purposes; so 14 MB for precise tool invocation is a reasonable memory footprint when a number of other local models are running (e.g. STT, TTS +…
yes, that's what we had in mind while building