81–90 of 175 posts
My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?
It's not a conversational model. It's meant as a local tool calling model.
Funny result from the web demo. I'm well aware that it's an extremely small and, well, stupid, model, but even so: Query: HN Result: { "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 } I'd expect it to at least ignore (call no tools) for the queries that it doe…
The website says the model is for "tool calling, device use, and structured extraction". Your example just doesn't seem to be very relevant. FWIW, it did a pretty good job for tool calling when I tried it, and I think it could be pretty nice to have this running on locally and integrate with Home Assistant.
I am not sure how a micro model will fundamentally solve it. Would love to understand what dannyw and team did there?
My first query: > Make it a little warmer in here. The reply: > "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.", Maybe I'm doing it wrong?
It's not a conversational model. It's meant as a local tool calling model.
Earlier quoted context omitted.
The website says the model is for "tool calling, device use, and structured extraction". Your example just doesn't seem to be very relevant. FWIW, it did a pretty good job for tool calling when I tried it, and I think it could be pretty nice to have this running on locally and integrate with Home Assistant.
I think the test above is about tool calling... That's how I read it. The issue here is known as "out of distribution detection" in the old-timey classification world. I am not sure how a micro model will fundamentally solve it. Would love to understand what dannyw and team did there?
Earlier quoted context omitted.
14mb? More like sub-$20 devices.
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
> turn on the tv { "function_calls": [ { "name": "lock_door", "arguments": { "door": "tv" } } ], "confidence": 0.0158 } Very interesting, seems confidence is 0 when tool calls are right?
Is there a prebuilt apk somewhere?
but if you're just looking for somewhere to try the model, use our in-browser playground! - https://cactuscompute.com/needle
This is really cool, I'm curious how much knowledge can their be in smaller models? It seems the current trade off is you need sizeably larger models for more performance but I'm curious if in your work how far this is true, as edge ai is really what needs to get better before physical ai can take off (my two cents).
I imagine at such a low parameter count, there would be little to no world knowledge whatsoever, and the entire focus is on getting the structure of tool calling etc. right…? But yeah, in terms of “physical” AI, robotics definitely comes to mind for me as well, where tool calls/structured “device” use in a “realtime”/edge application are highly beneficial (if you wanted to go with LLMs), but beefy hardware can’t be e…
yes you're right, there's only so much a 14MB model can do.
Needle excels at in-conext inference, with tightly defined environments. In our experience:
accurate descriptions + narrow tool scope = success