Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

cactuscompute.com

Post reply on HN
Print view View original

171–180 of 214 posts

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#172

Post by peaknk »

This is great, I have tried a couple different ways to hook up my Home assistant with Openclaw, Maybe my claw is bloated but a custom agent implementation which just has access to HA tool calling performs way better, although it costs money(Claude API). Would love to give this model a try and see how it performs.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#173

Post by wky »

cyprianaa263 wrote: post #169

[dead]

Considering it as a classification problem, you could use a representative set of example queries, feed them to this model, then ask a "smart" language model to assess each query + result for whether the result is actually correct. Then you have a dataset you can compute FPR and FNR for any given threshold, and score based on the context, ex. turning the lights on or off is a lot less important if it is right compared to whether your door is locked. You could even pick it based on the tool call itself: Low threshold for locking the door, high threshold for unlocking the door.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#174

Post by wky »

Specifying units seems to be unreliable; I tried adding a description to the set_thermostat temperature:

    "temperature": {
      "type": "number",
      "description": "degrees Fahrenheit"
    },
Set the living room temperature to 70 degrees Celsius

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70,
            "mode": "cool"
          }
        }
      ],
      "confidence": 0.6045
    }
Set the living room temperature to 70 degrees Fahrenheit

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70,
            "mode": "heat"
          }
        }
      ],
      "confidence": 0.4536
    }
Set the living room temperature to 70 degrees

    {
      "function_calls": [
        {
          "name": "set_thermostat",
          "arguments": {
            "room": "living room",
            "temperature": 70
          }
        }
      ],
      "confidence": 0.8517
    }
Trying "in degrees Fahrenheit" for the tool description had similarly counterintuitive confidences.

Edit: to be clear, the counterintuitive behavior is that the confidence ended up higher for the wrong units.

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#175

Post by victor_edka »

dbeardsl wrote: post #74

My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?

but wouldn't it need to know the current temp for that to work? my understanding was that the model is best at tool calling

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#178

Post by profsummergig »

genxy wrote: post #75
profsummergig wrote: post #61

Earlier quoted context omitted.

That sounds like an enormously expensive exercise.

You can train a model of this size on your laptop in a day.

So is the challenge, now, getting the right sort of data?

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#179

Post by semuggaemmanuel »

This is so cool but for me safety remains my priority first in place. So, if it violates my registered rules, and does what it wasn't supposed too, then what next.

Today AI agents are becoming difficult to safety most especially in critical sectors like banking, a mere mistake may lead to loss of cash or life if it's a hospital

My question is how safe is it for critical sectors

Re: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

#180

Post by bottlero_cket »

When I asked it to call my cable company I got this response. I am wondering how this response comes out… why does it say lights twice and not mention door? LLMs are a mystery to me but I suppose this reasoning is at the heart of the model and it’s only 40% confident in it

  {
  "function_calls": [],
  "reasoning": "No tool for making phone calls or initiating a call. The available tools are limited to smart home lights, thermostat, and lights.",
  "confidence": 0.4096
  }
Post reply on HN