what does the first L mean in LLM?
it stands for Lets-not-be-sarcastic :)
Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
91–100 of 188 posts
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#92Earlier quoted context omitted.
It's not a conversational model. It's meant as a local tool calling model.
Yes, I think OP understands that. What he and many others in this thread are trying to understand is what makes this model useful.
Turning a voice command into a tool call should be self evidently useful, being able to do that on a small embedded device is the novelty here. In this theoretical example, the thermostat is hosting the model on device and would use no external services.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#93Earlier quoted context omitted.
Yes, I think OP understands that. What he and many others in this thread are trying to understand is what makes this model useful.
Turning a voice command into a tool call should be self evidently useful, being able to do that on a small embedded device is the novelty here. In this theoretical example, the thermostat is hosting the model on device and would use no external services.
I confused by the dispatch model. Tool calls typically need some reasonability to be deterministic and, more importantly, predictable in response (o/w GIGO). Why would the thermostat need to interpret a voice command that the node capturing the voice command would not interpret instead?
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#94If you want Needle2 to rget lots of testing, become well known, etc - make a Home Assistant plugin.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#95This is so cool! Congrats to the team!
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#96My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?
Try asking it to set the thermostat to a value. It's a very small model, there's not much reasoning capability
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#97Earlier quoted context omitted.
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
Well running from QSPI flash (even the internal memory versions use SPI internally) so any inference would be very slow streaming from that compared to RAM. The featured article says: “With a peak session RAM around 28MB, Needle runs on newer microcontrollers like ESP32-S3.” So I don’t see this doing anything useful on a Pico. The Pico 2 (RP2350) for example has 520k of RAM.
An ESP32 has the same amount of SRAM as the Pi Pico. You can hook up PSRAM to the Pi Pico just like ESP32 to get 16MB more RAM.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#98these micro LLMs could be a game changer for hearing aids
so many interesting lowfi hardware use cases
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#99Its pretty significant you've got this working locally in wasm. Very cool.
Re: robotics: I'm unsure how this could be helpful.
It fails a pretty simple navigation prompt.
X0: (0.0, 0.0). Object bounding box: [1.0, 1.0, 2.0, 2.0]. navigate to (3.0,3.0)
I changed it to "call path planner to navigate: a_star(x0, xf, obs)"
Another fail.
My intuition tells me micro llms will/are important for robotics. I just can't grok it. Can someone without control theory experience give me a good example?
Probably at the planning level of the navigation stack. That's where I see reasoning being helpful. Lower than that...idk
Give me an example of a robotics prompt that seems useful and I'll give you an example why we don't need LLMs to be a tracking controller, etc.
Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
#100Earlier quoted context omitted.
Turning a voice command into a tool call should be self evidently useful, being able to do that on a small embedded device is the novelty here. In this theoretical example, the thermostat is hosting the model on device and would use no external services.
I confused by the dispatch model. Tool calls typically need some reasonability to be deterministic and, more importantly, predictable in response (o/w GIGO). Why would the thermostat need to interpret a voice command that the node capturing the voice command would not interpret instead?
The node capturing the voice command could be the thermostat. From my understanding they are targeting very small devices.
So that could be a master home automation node, but why not also a single purpose device? I can think of more bad examples than I can good ones, but maybe I am doing some soldering and I need my soldering iron turned up a bit; my hands are full, so doing that by voice would be useful enough. Something I would never link up to a big AI model or home automation network, but could be useful to control by voice.
If it's something that can be burnt directly into a chip and shipped with the products for cheap, maybe that's a more pragmatic way to get AI into small devices (see taalas for a much bigger model doing that, althoug not yet cheap).