Awesome! I just tried to set an alarm and add some groceries to the shopping list, and it outperformed Siri.
Music to our ears!
Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
71–80 of 255 posts
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#72This would be amazing for home assistant.
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#73Dumb questions, from someone not in the field...
What is a distilled model?
Why doesn't Google do this (to make their models smaller)?
Seems like you could make a competitor to Gemini?
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#74Dumb questions, from someone not in the field... What is a distilled model? Why doesn't Google do this (to make their models smaller)? Seems like you could make a competitor to Gemini?
No question is stupid!
1. Distilled means taking the intelligence of a big model and compacting into a tiny model.
2. Google already does so with FunctionGemma, but Needle argues that better performance could be achieved with 10x smaller model using our technologies.
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#75Dumb questions, from someone not in the field... What is a distilled model? Why doesn't Google do this (to make their models smaller)? Seems like you could make a competitor to Gemini?
Model distillation is lossy compression of big model to produce a smaller model.
Smaller model requires less space on disk, less video memory, and less compute (cheaper hardware).
Downside is that distilled model performs worse on the same benchmarks compared to original model.
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#76This is pretty much exactly what I want for Home Assistant. I yell out, "Computer! Lights!" and it toggles the lamp in the room on or off. (I mean I can do that now, I think, but probably with a much larger model.) I haven't played with it yet, but does it ever return anything other than a tool call? What are the failure modes? What if it doesn't understand the request? Does it ever say it can't find a tool? Does it…
How many lights are there?
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#77Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#78Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#79This would be amazing for home assistant.
On my list to check out tomorrow :D
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#80I don't really understand what this is for... there is a lot of ML-researcher talk on the GH page about the model architecture, but how should I use it? Is it a replacement for Kimi 2.7, Claude Haiku, Gemini Flash 3.1 lite, a conversational LLM for the situations where it's mostly tool-calling like coding and conversational AI?
It is for building agentic capabilities into very small devices like phones, glasses, watches and more. Does that make sense?
[flagged]