Suggestion: publish a live demo of the "needle playground". It's small enough that it should be pretty cheap to run this on a little VPS somewhere!
Should be quick and easy with WebGPU, too.
Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
201–210 of 255 posts
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#202This is really cool to see the push for tiny models
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#203Can this be converted to onnx or otherwise be used in a browser?
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#204What Google's response will be to this.
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#205[flagged]
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#206FYI, distilling Gemini is explicitly against the ToS: "You may not use the Services to develop models that compete with the Services (e.g., Gemini API or Google AI Studio). You also may not attempt to reverse engineer, extract or replicate any component of the Services, including the underlying data or models (e.g., parameter weights)."
...unless you are trying to get Google's attention for a job interview. :)
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#207[flagged]
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#208This seems to be something that could be plugged into an Alexa style device? You had 15 tool categories, what were they and will it work on categories outside of this 15?
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#209Earlier quoted context omitted.
Thanks to a Huggingface linked below, I tested it and im not impressed. prmopt: i need to contact my boss i will be late. Result: 20mins [{"name":"set_timer","arguments":{"time_human":"20 minutes"}}]. It didnt use the email tool and i tried 2-3 different ways of asking it.
Query: context: { "boss_email": "bigboss69420@corporatepersonhood.net", "upcoming_meetings": [{ with: "bigboss69420@corporatepersonhood.net", "time": "11:00" }] } user: i need to contact my boss i will be late, could you tell him I'll be 15 minutes late? Output: [{"name":"send_email","arguments":{"to":"bigboss69420@corporatepersonhood.net","subject":"upcoming_meetings","body":"I'll be 15 minutes late"}},{"name":"send…
Maybe it has to be fine-tuned per tool just like functiongemma
Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
#210Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Although I know people will object to including an extra 14 MB and the computation for "parsing" and it could be pretty bad if everyone started doing that. But it's really interesting to me that that may be possible now. You can include a fine-tuned model that under…
I wonder if it will make sense to have a central natural language parser of sorts managed by the OS that includes a model like this? So all command line programs can hook into this model at runtime, and you can have adaptors for fine-tuning etc.
Apple already have this working on iOS - as discussed in this recent post.