Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

github.com

Post reply on HN
Print view View original

211–220 of 255 posts

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#211

Post by isityettime »

AndrewKemendo wrote: post #175
nl wrote: post #99

Do you have any examples or data on the discriminatory power of the model for tool use? The examples are things like "What is the weather in San Francisco", where you are only passed a tool like tools='[{"name":"get_weather","parameters":{"location":"string"}}]', I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs. My question is how effective is it at handling…

I’m noticing a trend where people who have no experience with good old-fashioned AI are starting to learn about it, and use it to save money on their tool chain costs. I think it’s great that people are finally rediscovering these basics and maybe at some point they’ll realize that AI is not something new

In much the same way, LLMs are renewing interest in general automation that also falls short of GOFAI. In some situations, I think this might be very beneficial.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#212

Post by fwip »

mahmoudimus wrote: post #117
michelsedgh wrote: post #112

Earlier quoted context omitted.

Thanks to a Huggingface linked below, I tested it and im not impressed. prmopt: i need to contact my boss i will be late. Result: 20mins [{"name":"set_timer","arguments":{"time_human":"20 minutes"}}]. It didnt use the email tool and i tried 2-3 different ways of asking it.

works for me: input: i need to contact my boss i will be late. output: [{"name":"send_email","arguments":{"to":"boss@company.com","subject":"Running late","body":"I will be late for the meeting."}}] it did have the send_email tool on the left hand side though

If it were me, I wouldn't be happy that it made up a meeting that I didn't mention.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#214

Post by [deleted] »

michelsedgh wrote: post #112
nl wrote: post #99

Do you have any examples or data on the discriminatory power of the model for tool use? The examples are things like "What is the weather in San Francisco", where you are only passed a tool like tools='[{"name":"get_weather","parameters":{"location":"string"}}]', I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs. My question is how effective is it at handling…

Thanks to a Huggingface linked below, I tested it and im not impressed. prmopt: i need to contact my boss i will be late. Result: 20mins [{"name":"set_timer","arguments":{"time_human":"20 minutes"}}]. It didnt use the email tool and i tried 2-3 different ways of asking it.

[deleted]

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#217

Post by bckr »

binyang_qiu wrote: post #139

A lot of agent workflows really are just tool selection + argument extraction + structured output. How does this behave once workflows become multi-step and state starts accumulating across calls?

I think it’s worth having a tiny model that can convert a short sentence into a structured tool call. Now you can add an orchestration layer and you have separation of concerns. That orchestration layer can be a mixture of deterministic and probabilistic systems. Break the foundation model into several single use tools, please!

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#218

Post by insumanth »

Smaller tool calling models are absolutely needed when we are trying to apply agentic workflow to on-device, small systems, where data is private.

I currently use small general purpose LLM. But a special finetune is always the way. I'll try this for my use case.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#219

Post by varenc »

madduci wrote: post #134
varenc wrote: post #133

Are you worried about Google's response to this? Google reportedly reacts to distillation attempts " with real-time proactive defenses that can degrade student model performance ". So if they detected you, they could have intentionally fed you a dumber but plausible variant of Gemini: https://cloud.google.com/blog/topics/threat-intelligence/dis... But also, this model is small and just focusing on the tool use. In te…

Well, it's like robbing the robbers, when it comes to training data

This perspective is more cut and dry when its someone like OpenAI scraping the whole internet explicitly for LLM training purposes. But Google has already been scraping the entire internet for 25+ years. At what point did building a smarter search engine transition from indexing, to 'robbing'? And it's not like training Gemini is the first time they used their internet cache to build AI. AI, as academics use the term, has been in use on Google results for a long time.

Basically, if we were okay with Google scraping the internet to build a search index, what is the line they crossed that turned this from acceptable search engine indexing, into theft?

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#220

Post by tim-projects »

This sounds like it could be really powerful, but I don't understand how to use it.

I tried 'git add commit and push' which is something I might ask an llm to do but is a waste of tokens.

It output:

[{"name":"create_note","arguments":{"text":"commit and push","title":"git add"}}]

Which is kind of a little way there but not useable on its own

Post reply on HN