Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

cactuscompute.com

Post reply on HN
Print view View original

171–175 of 175 posts

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#172

Post by peaknk »

This is great, I have tried a couple different ways to hook up my Home assistant with Openclaw, Maybe my claw is bloated but a custom agent implementation which just has access to HA tool calling performs way better, although it costs money(Claude API). Would love to give this model a try and see how it performs.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#173

Post by wky »

cyprianaa263 wrote: post #169

A small on-device model with escalation is compelling. How do you decide when confidence is low enough to hand off to a larger model?

Considering it as a classification problem, you could use a representative set of example queries, feed them to this model, then ask a "smart" language model to assess each query + result for whether the result is actually correct. Then you have a dataset you can compute FPR and FNR for any given threshold, and score based on the context, ex. turning the lights on or off is a lot less important if it is right compared to whether your door is locked. You could even pick it based on the tool call itself: Low threshold for locking the door, high threshold for unlocking the door.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#174

Post by wky »

Specifying units seems to be unreliable; I tried adding a description to the set_thermostat temperature:

    "temperature": {
      "type": "number",
      "description": "degrees Fahrenheit"
    },
Set the living room temperature to 70 degrees Celsius

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70,
            "mode": "cool"
          }
        }
      ],
      "confidence": 0.6045
    }
Set the living room temperature to 70 degrees Fahrenheit

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70,
            "mode": "heat"
          }
        }
      ],
      "confidence": 0.4536
    }
Set the living room temperature to 70 degrees

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70
          }
        }
      ],
      "confidence": 0.8517
    }
Trying "in degrees Fahrenheit" for the tool description had similarly counterintuitive confidences.

Edit: to be clear, the counterintuitive behavior is that the confidence ended up higher for the wrong units.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#175

Post by victor_edka »

dbeardsl wrote: post #74

My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?

but wouldn't it need to know the current temp for that to work? my understanding was that the model is best at tool calling
Post reply on HN