Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

cactuscompute.com

Post reply on HN
Print view View original

51–60 of 175 posts

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#51

Post by yorwba »

"make it as dark as possible"

  {
    "function_calls": [
      {
        "name": "set_thermostat",
        "arguments": {
          "temperature": 72,
          "mode": "cool",
          "room": "living room"
        }
      }
    ],
    "reasoning": "'as dark as possible' -> set_thermostat to warm; 'dark' implies higher temperature; 'cool' mode for darkness.",
    "confidence": 0
  }
... maybe this counts as dark humor at least.

Since it seems limited to matching a few templates and otherwise falling flat on its face, I wonder how 14MB of regexes would fare in its stead. Normally you wouldn't want to parse arbitrary natural language input with regex because of how tedious and brittle it would be, but for the tedium we have LLMs and this alternative isn't exactly robust either.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#52

Post by HenryNdubuaku »

hathym wrote: post #47
hathym wrote: post #45

Earlier quoted context omitted.

works better that way, thanks :)

but still struggle when changing the quesion: import needle @needle.tool def add(a: int, b: int): "Calculate the sum of two numbers. Use for any arithmetic or math question." return a + b agent = needle.Needle(tools=[add]) print(agent.run("what is 5 + 7?")["reasoning"]) >> No calculator or math tool available. Cannot compute numbers.

Ah, another failure point on our end! So a simple "5 + 7" and "add 5 and 7" works. But to handle ambiguity, the python package ships pipelines to synthesize augmentations and fine-tune on your samples for robustness. Just run "needle playground" and use the UI. Apologies.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#53

Post by HenryNdubuaku »

skavi wrote: post #50

I wonder if there's any way to get this to plan out a dag of tool calls? i.e. use the results from earlier calls as the parameters to later ones? I tried introducing a stack based system, but gave up pretty quickly.

Yes, though for better results in production, after creating your tool json, use the provided data synthesis and fine-tuning pipeline. It tunes on on your mac.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#54

Post by rshemet »

forsalebypwner wrote: post #41

Any instructions available for running this on an ESP32-S3 or P4 like the site says?

Hey! Roman here from Cactus - yes, we're putting putting together a detailed guide for ESP32.

In the meantime, if you have enough RAM for the current model (≈28MB), our repo will get you up & running:

https://github.com/cactus-compute/needle

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#56

Post by dalemhurley »

redrix wrote: post #26

This is cool! While most of the industry focuses on the frontier of “intelligence” (function), a release like this represents the frontier of the other end of the spectrum (form). Both are important if we ever want to see “Opus-level” capability running locally on commodity machines in the future.

agreed, this is where we have the biggest opportunity for innovation.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#57

Post by plingbang »

Tiberium wrote: post #4

Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…

I've got an identical output with the prompt "do not lock the door".

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#59

Post by hgoel »

profsummergig wrote: post #39

Could someone please share how such open source micro-LLMs might have been created? Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?

Another option for something this small and narrowly specialized could be to get traditional LLMs to synthesize the training data. Model collapse is probably less of an issue at this size relative to terabyte sized models.
Post reply on HN