Live data from Hacker News

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

cactuscompute.com

151–160 of 188 posts

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#151
post #4

Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…

It seems to love locking doors. I was hoping this would dim the lights. > I'm hungover { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. 'hungover' implies a security door. No specific door named, so use 'front door' as default.", "confidence": 0 }

Cannot unlock, though:

Prompt: "Unlock my doors"

{ "function_calls": [], "reasoning": "No tool for unlocking or locking door exists. lock_door only performs lock/unlock actions, not unlock.", "confidence": 0.9492 }

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#152

This is cool. I definitely think the "micro" sized LLM space is underappreciated, so it's always good to see work like this. I foresee a paradigm in some contexts where you have a hierarchy of LLMs, with more competent models actively training smaller models to solve specific tasks very efficiently, and something like this could be the smallest layer in that stack. With that being said, the web demo is not particular…

It's called a LM :). LLM stands for Large Language Model.

LLM as in Little Language Model

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#154
Looks incredibly cool, although it ran at ~4 tok/s on my browser (I assume because I've disabled V8). I believe your numbers though. I always wanted to try using something like this to create an AI assistant for my phone. If you try to create something without AI, you'll quickly run into the problem of using NLP to map input to the equivalent of AI tool calls. On the other hand, shelling out to a full-blown LLM for every input is slow, costly and if you want it local, greatly restricts its computing power and the number of devices it can run on. Needle and similar models solve this problem nicely: they're incredibly cheap and handle NLP (not in a predictable way of course, but with great success rates), so you can quickly run predefined actions or shell out to a proper LLM for more complicated tasks. You have the best of two worlds.

You may have inspired me to give it a try!

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#155

This is cool. I definitely think the "micro" sized LLM space is underappreciated, so it's always good to see work like this. I foresee a paradigm in some contexts where you have a hierarchy of LLMs, with more competent models actively training smaller models to solve specific tasks very efficiently, and something like this could be the smallest layer in that stack. With that being said, the web demo is not particular…

It's called a LM :). LLM stands for Large Language Model.

[deleted]

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#158

Looks incredibly cool, although it ran at ~4 tok/s on my browser (I assume because I've disabled V8). I believe your numbers though. I always wanted to try using something like this to create an AI assistant for my phone. If you try to create something without AI, you'll quickly run into the problem of using NLP to map input to the equivalent of AI tool calls. On the other hand, shelling out to a full-blown LLM for e…

Please give it a try and let us know!

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#159

If you want Needle2 to rget lots of testing, become well known, etc - make a Home Assistant plugin.

hey Kenny, Roman from Cactus here - could you say more? What kind of home assistant / what stack

I think they're talking about you making Cactus Needle be an engine to respond to user queries in Home Assistant by making a custom add-on.

https://www.home-assistant.io/voice_control/best_practices/

https://developers.home-assistant.io/docs/core/llm/

https://developers.home-assistant.io/docs/core/entity/conver...

Something like the Ollama integration:

https://www.home-assistant.io/integrations/ollama/

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#160

Looks incredibly cool, although it ran at ~4 tok/s on my browser (I assume because I've disabled V8). I believe your numbers though. I always wanted to try using something like this to create an AI assistant for my phone. If you try to create something without AI, you'll quickly run into the problem of using NLP to map input to the equivalent of AI tool calls. On the other hand, shelling out to a full-blown LLM for e…

310 tok/sec on my browser, hm.
Post reply on HN