Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

github.com

Post reply on HN
Print view View original

171–180 of 255 posts

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#171

Post by _flux »

ai_fry_ur_brain wrote: post #137
jumploops wrote: post #132

This is neat, and matches an observation I saw with early Claude Code usage: Sonnet would often call tools quickly to gather more context, whereas Opus would spend more time reasoning and trying to solve a problem with the context it had. This led to lots of duplicated functions and slower development, though the new models (GPT-5.5 and Opus 4.6) seem to suffer from this less. My takeaway was that “dumber” (i.e. smal…

The key is to not run LLMs in loops. This trend of agentic frameworks is silly, and mostly exists to make LLM companies more revenue. An LLM is mostly useless but is much more useful and reliable with one shot tooling. I have a suite or tools ive built for myself on top of the openrouter api for very specific tasks. Press button amd LLM does (one) useful thing, not press button and let LLM run tool calls in a loop fo…

Often I find LLMs doing multiple steps to achieve some goals (e.g. do certain operations against JIRA or Gitlab), and if the LLM work seems useful, I instruct it to create a tool to achieve the task more directly and revise skill data to make use of the tool.

Granted I've let it mostly vibecode those tools, so they might be garbage. I should perhaps have it do a refactoring round to make more composable tools..

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#173

Post by TobTobXX »

giancarlostoro wrote: post #52
bityard wrote: post #50

Earlier quoted context omitted.

Since there is no GUI here, I feel like a simple plaintext chat transcript would be both 100x smaller and 100x easier to read. (Not to mention accessible.)

Sure, and we've seen those terminal screen recorders that give you back a replayable demo, that could work too.

https://asciinema.org/ in case anyome is searching it.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#174

Post by TobTobXX »

meander_water wrote: post #140

I'm so excited for this, nice work! Gemma4 edge models were promised to be great for agentic use, but have been really disappointing in all my tests. They fail at the most basic tool use scenarios. Have you run any tool-use benchmarks for Needle, or do you plan to? Would be great if you could add results to the repo if so.

Wait what? I've used DeepSeek V4 flash a lot and compsred to Gemma 4 E2B (ie. the smallest, event at q4), it consistently underperforms. In contrast to DS flash, I've found Gemma 4 to be incredibly precise and consistent with tool use.

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#175

Post by AndrewKemendo »

nl wrote: post #99

Do you have any examples or data on the discriminatory power of the model for tool use? The examples are things like "What is the weather in San Francisco", where you are only passed a tool like tools='[{"name":"get_weather","parameters":{"location":"string"}}]', I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs. My question is how effective is it at handling…

I’m noticing a trend where people who have no experience with good old-fashioned AI are starting to learn about it, and use it to save money on their tool chain costs.

I think it’s great that people are finally rediscovering these basics and maybe at some point they’ll realize that AI is not something new

Re: Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model

#177

Post by matchaonmuffins »

ilaksh wrote: post #5

Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Although I know people will object to including an extra 14 MB and the computation for "parsing" and it could be pretty bad if everyone started doing that. But it's really interesting to me that that may be possible now. You can include a fine-tuned model that under…

I wonder if it will make sense to have a central natural language parser of sorts managed by the OS that includes a model like this?

So all command line programs can hook into this model at runtime, and you can have adaptors for fine-tuning etc.

Post reply on HN