Does Pi even need read/write/edit tools? Couldn't it just have bash commands and get the model to use e.g. sed for everything?
Better Models: Worse Tools
21–30 of 90 posts
Re: Better Models: Worse Tools
#22[flagged]
Re: Better Models: Worse Tools
#23Does Pi even need read/write/edit tools? Couldn't it just have bash commands and get the model to use e.g. sed for everything?
They do use these tools but they are not as efficient as codex multi-file patch which can perform file move, and edit in a single generation.
Re: Better Models: Worse Tools
#24It's not the failed call that worries me. The call itself was correct, and the only thing off was a couple of invented fields. That makes the runtime feel like part of the model's interface rather than just an implementation detail. Train a model in a forgiving environment and other runtimes end up inheriting its habits.
Re: Better Models: Worse Tools
#25Hey, an article right up my alley! AI infrastructure/tools engineer here (hic-ai.com); my flagship product, HIC Mouse, is a precision-editing system for coding agents designed to work across a wide array of models and harnesses. Mouse provides 11 tools exposed via MCP for read-, find-, and edit-operations, using a coordinate-based schema (as well as exact and multiple string replacement), a Dialog Box inspect/refine/…
Very cool tool. As the "moar tokens" era is starting to wind down I think people are going to realize just how crappy these harnesses really are, especially Claude Code.
I have gone back and forth between Claude and Cursor and it is clear Claude just throws the kitchen sink at problems to get an edge. I write MCP tools and I see these exact problems when the inputs and outputs aren't clearly defined, the LLM just guesses and retries.
Re: Better Models: Worse Tools
#26This is easily solved with good error messages.
Claude always gets the syntax wrong on my tool calls.
So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool.
The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It happens every session, but it only happens once per context window. After that the agent holds on to the lesson.
To do this for your own tool calls, imagine what you’d do in the agent’s place - what info you’d need so you can correct your mistake. Assume the agent wants to achieve the goal so it’ll try again. These are probabilistic systems, so we need to give them an extra loop to get the deterministic bits right.
Re: Better Models: Worse Tools
#27Yep. I spotted the same thing in piclaw (which relies on the pi runtime) but did not have time/energy to do a lot about it—and fable does the same, as far as I can tell, with one out of five or six edits failing. But I prefer OpenAI models for coding, so it wasn’t a real problem.
Re: Better Models: Worse Tools
#28I guess we are going to get even more of this. Where models and tools start producing nonsensical results and no-one understands why it appends and we must read articles like this that catch it.
Re: Better Models: Worse Tools
#29This is easily solved with good error messages. Claude always gets the syntax wrong on my tool calls. So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool . The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It happens every session, but it only happens once per context window. After that the agent holds on to the lesson. To d…
LSPs and linters serve the same purpose. I use the latter in git hooks.
Re: Better Models: Worse Tools
#30[flagged]