Live data from Hacker News

Better Models: Worse Tools

lucumr.pocoo.org

71–80 of 90 posts

Re: Better Models: Worse Tools

#71
There's a spectrum of possible explanation, from "this is a model training artifact which for now they correct via the harness" through to "this is deliberate, and creates a constantly moving target to make third-party harnesses less efficient for lock-in purposes".

I'd not discount the adversarial end of the spectrum.

Re: Better Models: Worse Tools

#72
post #39

Earlier quoted context omitted.

This will cause an extra round trip to the LLM. Which means more $ spent.

So? What alternative do you suggest? Let the LLM get it wrong forever? Remove the tool? Automatically try to patch the syntax? Almost no "solutions" in engineering/programming comes for free, one way or another, it's all a balancing act between different solutions with different tradeoffs. In this case, another request/response seems preferable to the other tradeoffs.

Maybe I'm dumb but I believe I can think of one totally free solution here..

Re: Better Models: Worse Tools

#73
Different but related: When you use a Codex subscription in an agent like Pi or OpenCode, all the requests and tool call execution go through a sandbox owned by Codex app server, and all the tool calls function somewhat differently, and you can't read files outside of the sandbox as easily. It's currently tripping me up a bit when building a model router.

Re: Better Models: Worse Tools

#74

Different but related: When you use a Codex subscription in an agent like Pi or OpenCode, all the requests and tool call execution go through a sandbox owned by Codex app server, and all the tool calls function somewhat differently, and you can't read files outside of the sandbox as easily. It's currently tripping me up a bit when building a model router.

> When you use a Codex subscription in an agent like Pi or OpenCode, all the requests and tool call execution go through a sandbox owned by Codex app server

That is not the case. There are some subtle differences between subscription and regular inference API, but not to the degree that behaviors change entirely. In Pi we're doing tests against both API and subscription API regularly to see how they behave.

Re: Better Models: Worse Tools

#76
post #45

This is easily solved with good error messages. Claude always gets the syntax wrong on my tool calls. So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool . The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It happens every session, but it only happens once per context window. After that the agent holds on to the lesson. To d…

I've built a library that makes creating rich feedback systems easier, check this out: https://tool2agent.org/

very cool.

Re: Better Models: Worse Tools

#77

This is easily solved with good error messages. Claude always gets the syntax wrong on my tool calls. So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool . The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It happens every session, but it only happens once per context window. After that the agent holds on to the lesson. To d…

So, is this part of the tool definition, or did you create your own coder agent?

Re: Better Models: Worse Tools

#78
post #39

Earlier quoted context omitted.

This will cause an extra round trip to the LLM. Which means more $ spent.

So? What alternative do you suggest? Let the LLM get it wrong forever? Remove the tool? Automatically try to patch the syntax? Almost no "solutions" in engineering/programming comes for free, one way or another, it's all a balancing act between different solutions with different tradeoffs. In this case, another request/response seems preferable to the other tradeoffs.

The counter intuitive pattern I see emerging is if you can cleanly determine intent, of the call you fix the call and prepend informative text to the tool call response indicating the mistake made and how to fix in the future then followed by the actual tool call. In this case you can validate fields and rather than throw a hard error determine if it's an extra field that isn't needed. If so you correct the call and prepend a corrective response in the tool call. This saves turns, it instructs the model in context so less likely to happen later and helps models that aren't so good at recovering from bad tool calls and staying on their longer horizon agentic task (most non openai and anthropic models)

Re: Better Models: Worse Tools

#80
post #75

I want dark window chrome and light contents but browsers seem completely unwilling to let me have this option.

If you search "how to turn on dark mode" in chrome, it'll change to dark mode. Or you can change it in settings. For the numerous websites that don't care, extensions exist to force them. Midnight lizard is my favorite.

Edge has it in edge://flags no extension needed. Firefox & brave requires settings and extension like chrome. I haven't used opera or Vivaldi in ages to help with them, but they will have an option because customizability is a key part of their selling points

Post reply on HN