Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

cactuscompute.com

Post reply on HN
Print view View original

121–130 of 188 posts

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#121

Post by havercosine »

Congratulations. 28MB is impressive, I've not played around with actual queries/outputs.

I'm wondering what is the overall thesis/plan here and where exactly the innovation lies? Would love if you can throw light on below,

  - If I understand, this is complete stack of a custom architecture (attention only transformers), custom quantisation format and a runtime engine all packaged together?
  - How do you differentiate / compete against LiteRT (former TensorFlowLite) and Lite RT LM? Google is heavily investing in this ecosystem because Android is where they have distribution moat. Wouldn't it be easier for me as a developer to build on top of LiteRT since it is relatively open ecosystem and I can pack large number of open models from HF directly?
  - What exact challenges you saw with TFLite, TVM etc that prompted this effort ?
  - What will be the pricing model like?

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#122

Post by Robin_Message »

> Turn the lights down low in the bedroom

Sets lights to 30% but also off

> Turn the lights low in the bedroom

Sets lights to on

This is a cool idea but I think humans assume more than 14MB of intelligence. This is like the unhelpful guard in the swamp castle of Monty Python's Holy Grail

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#123

Post by weird-eye-issue »

dannyw wrote: post #79
dbeardsl wrote: post #74

My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?

It's not a conversational model. It's meant as a local tool calling model.

That's exactly what they were testing and what it failed at

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#124

Post by HenryNdubuaku »

havercosine wrote: post #121

Congratulations. 28MB is impressive, I've not played around with actual queries/outputs. I'm wondering what is the overall thesis/plan here and where exactly the innovation lies? Would love if you can throw light on below, - If I understand, this is complete stack of a custom architecture (attention only transformers), custom quantisation format and a runtime engine all packaged together? - How do you differentiate /…

So TFLite is an inference engine while Needle is a model, which you can port on TFLite btw.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#125

Post by HenryNdubuaku »

Robin_Message wrote: post #122

> Turn the lights down low in the bedroom Sets lights to 30% but also off > Turn the lights low in the bedroom Sets lights to on This is a cool idea but I think humans assume more than 14MB of intelligence. This is like the unhelpful guard in the swamp castle of Monty Python's Holy Grail

Haha, its not a generalist model, cheap devices cannot run LLM in production, so we brig on-device intelligence to them in the size they can. Needle is niche, not a Gemma or GPT replacement.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#126

Post by HenryNdubuaku »

r0ze-at-hn wrote: post #116

Curious, why did you go down to 2bit rather than 4 bits? 4bit with folding the layers should arrive at the same size, but with better quality?

A lot of Needle 1 users deployed on microcontrollers, we couldn't just throw them under the bus only t chase benchmarks, so its a price we are willing to pay :(

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#127

Post by Robin_Message »

HenryNdubuaku wrote: post #125
Robin_Message wrote: post #122

> Turn the lights down low in the bedroom Sets lights to 30% but also off > Turn the lights low in the bedroom Sets lights to on This is a cool idea but I think humans assume more than 14MB of intelligence. This is like the unhelpful guard in the swamp castle of Monty Python's Holy Grail

Haha, its not a generalist model, cheap devices cannot run LLM in production, so we brig on-device intelligence to them in the size they can. Needle is niche, not a Gemma or GPT replacement.

Right, but I don't think consumers would consider this "intelligence". It's too dumb.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#128

Post by HenryNdubuaku »

kooi wrote: post #99

Its pretty significant you've got this working locally in wasm. Very cool. Re: robotics: I'm unsure how this could be helpful. It fails a pretty simple navigation prompt. X0: (0.0, 0.0). Object bounding box: [1.0, 1.0, 2.0, 2.0]. navigate to (3.0,3.0) I changed it to "call path planner to navigate: a_star(x0, xf, obs)" Another fail. My intuition tells me micro llms will/are important for robotics. I just can't grok i…

[dead]
Post reply on HN