Each text token already represents the activation of certain neurons. There is nothing "more direct." And you cannot fully separate data and metadata if you want them to influence the output. At best you can clearly distinguish them and hope that this is enough for the model to learn to treat them differently.
Are there tokens reserved for tool calls? If yes, I can see the equivalence. If not, not so much.
Useful article, I was fighting with GLM's tool calling format just last night. Stripping and sanitization to make it compatible with my UI consistently has been... fun.
One of the most relevant posts about AI on HN this year. It's not hype-y, but it's imperative to discuss. I find it strange that the industry hasn't converged in at least somewhat standardized format, but I guess despite all the progress we're still in the very early days...
Sounds like we need another standard. /s This is one of the first tech waves where I feel like I'm on the very very groundfloor for a lot of exploration and it only feels like people have been paying closer attention in the last year. I can't imagine too many 'standard' standards becoming a standard that quickly. It's new enough that Google seems to be throwing pasta against the wall and seeing what products and prot…
Depending on a vendors market position, they may not want to make it easy to switch, which is what standards do, no?
I guess I fail to see why this is such a problem. Yes it would be nice if the wire format were standardized or had a standard schema description, but is writing a parser that handles several formats actually a difficult problem? Modern models could probably whip up a "libToolCallParser" with bindings for all popular languages in an afternoon. Could probably also have an automated workflow for adding any new ones with…
Author here. You're right, it's not a hard problem, but a particularly annoying one.
I guess I fail to see why this is such a problem. Yes it would be nice if the wire format were standardized or had a standard schema description, but is writing a parser that handles several formats actually a difficult problem? Modern models could probably whip up a "libToolCallParser" with bindings for all popular languages in an afternoon. Could probably also have an automated workflow for adding any new ones with…
Maybe they could vibe code some sort of, I don't know, a Web Service Description Language. That could describe how to interact with a service.
MCP is the wire format between agent and tool, not the format the model itself uses to emit the call. That part (Harmony, JSON, XML-ish) is still model-specific. So the M×N the article describes is really two problems stacked — MCP only solves the lower half.
Also in practice Claude Code, Cursor and Codex handle the same MCP tool differently — required params, tool descriptions, response truncation. So MCP gives you the contract but the client UX still leaks.
The key part of the article is that token structure interpretation is a training time concern, not just an input/output processing concern (which still leads to plenty of inconsistency and fragmentation on its own!). That means both that training stakeholders at model development shops need to be pretty incorporated into the tool/syntax development process, which leads to friction and slowdowns. It also means that any current improvements/standardizations in the way we do structured LLM I/O will necessarily be adopted on the training side after a months/years lag, given the time it takes to do new-model dev and training.
That makes for a pretty thorny mess ... and that's before we get into disincentives for standardization (standardization risks big AI labs' moat/lockin).
The models only output text. Tool calls are nothing more than specially formatted text which gets parsed and interpreted by the inference server (or some other driver) into something which can be picked up by your agent loop and executed. Models are trained in a wide variety of different delimiters and escape characters to indicate their tool calls (along with things like separate thinking blocks). MCP is mostly a st…
> Tool calls are nothing more than specially formatted text which gets parsed and interpreted by the inference server I know this is getting off-topic, but is anybody working on more direct tool calling? LLMs are based on neural networks, so one could create an interface where activating certain neurons triggers tool calls, with other neurons encoding the inputs; another set of neurons could be triggered by the token…
I'm a novice in this area, but my understanding is that LLM parameters ("neurons", roughly?), when processed, encode a probability for token selection/generation that is much more complex and many:one than "parameter A is used in layer B, therefore suggest token C", and not a specific "if activated then do X" outcome. Given that, how would this work?
MCP is the wire format between agent and tool, not the format the model itself uses to emit the call. That part (Harmony, JSON, XML-ish) is still model-specific. So the M×N the article describes is really two problems stacked — MCP only solves the lower half. Also in practice Claude Code, Cursor and Codex handle the same MCP tool differently — required params, tool descriptions, response truncation. So MCP gives you…
But, like pancakes, usually the stack is described as building bottom-up. Can you relate the individual components to ingredients in a diner-style pancake breakfast?