Live data from Hacker News

Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

github.com

91–100 of 255 posts

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#91
post #69

Earlier quoted context omitted.

[flagged]

Pardon me, do I know you? Why are you attacking me?

I don't think they're attacking you, but suggesting you read more carefully. The information provided is correct and clear, but you need to let go of your own biases when consuming it.

I personally prefer the M to the B. I guess as an engineer, noticing the units comes pretty naturally.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#94
post #48

Can it summarize text it fetches? Come to think of it, this could be a nice model to have as the first pass in a more complex agent system where Needle hands of the results of a tool call to a larger model. I will defiantly play around with this!

> I will defiantly play around with this!

Are you Calvin or Hobbes?

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#95
post #53

This is pretty much exactly what I want for Home Assistant. I yell out, "Computer! Lights!" and it toggles the lamp in the room on or off. (I mean I can do that now, I think, but probably with a much larger model.) I haven't played with it yet, but does it ever return anything other than a tool call? What are the failure modes? What if it doesn't understand the request? Does it ever say it can't find a tool? Does it…

How many lights are there?

… four. There are four lights.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#96
post #64

From all the models that do toolcalls the only thing I am confused is why did you pick the worst? Or maybe they are only bad in agentic work it fine for one shot toolcalls?

Gemini is pretty solid for 1-shot tool call and affordable as well.

Hi, would love to know where you get that impression on 1 shot tool calling, was there concrete evaluation carried out? pretty new to this and was a bit lost when trying to compare models on different capabilities.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#98

That M versus B is way too subtle. 0.026B is my suggestion

The "M" nomenclature has been around since at least BERT and T5/FLAN. It's valid to use it even if today's LLM devs are more familiar with billion-scale models.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#99
Do you have any examples or data on the discriminatory power of the model for tool use?

The examples are things like "What is the weather in San Francisco", where you are only passed a tool like

  tools='[{"name":"get_weather","parameters":{"location":"string"}}]',
I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs.

My question is how effective is it at handling ambiguity.

Can I send it something like a text message "lets catch up at coffee tomorrow 10:00" and a command like "save this" and have it choose a "add appointment" action from hundreds (or even tens) of possible tools?

[1] https://github.com/nlothian/Acuitra/wiki/About

Post reply on HN