Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The M×N problem of tool calling and open-source models

thetypicalset.com

Post reply on HN
Print view View original

51–56 of 56 posts

Re: The M×N problem of tool calling and open-source models

#51

Post by casey2 »

Ironically LLMs solve the MxN problem he's complaining about. He wants to get rid of the problem entirely, but fails to see the value of pointless differences.

It's the same kind of hubris that asks why we don't all speak one language. In the future we will all speak one language and we will all speak either our own or a DSL shared by only a few others, in America we will all speak English, in Japan even the torists will all speak Japanese. Very few will know English, but some will know it better than anyone.

Re: The M×N problem of tool calling and open-source models

#52

Post by remilouf »

casey2 wrote: post #51

Ironically LLMs solve the MxN problem he's complaining about. He wants to get rid of the problem entirely, but fails to see the value of pointless differences. It's the same kind of hubris that asks why we don't all speak one language. In the future we will all speak one language and we will all speak either our own or a DSL shared by only a few others, in America we will all speak English, in Japan even the torists…

> Ironically LLMs solve the MxN problem he's complaining about

Enlighten me please

Re: The M×N problem of tool calling and open-source models

#53

Post by imtringued »

gertlabs wrote: post #45
airstrike wrote: post #3

One of the most relevant posts about AI on HN this year. It's not hype-y, but it's imperative to discuss. I find it strange that the industry hasn't converged in at least somewhat standardized format, but I guess despite all the progress we're still in the very early days...

In our benchmarks we exclusively use a custom harness for measuring tool capability. It has common tools that any harness would have, like a thin wrapper around shell commands, basic file editors, etc. but an important part of agentic intelligence is adapting to new tools. Frontier models are already quite adaptable, especially Anthropic models, and improving with each release. I think a standardized format will beco…

This is backwards. If you think the models are capable of adapting to any format, they will have an easier time adapting to more popular and more common formats until they will eventually become de-facto standards.

The only case where a standard wouldn't win is the case where models are only capable of supporting the baked in format but even this could be solved by adopting a standard format.

Re: The M×N problem of tool calling and open-source models

#54

Post by all2 »

nasreddin wrote: post #47
all2 wrote: post #38

I wonder if stuffing tool call formatting into an engram layer (see Deepseek's engram paper) that could be swapped at runtime would be a useful solution here. The idea would be to encode tool calling semantics once on a single layer, and inject as-needed. Harness providers could then give users their bespoke tool calling layer that is injected at model load-time. Dunno, seems like it might work. I think most open sou…

The engram idea is actually technically clever but imo sees the solution from a bottom-up approach while Louf's real argument is a top-down view. His solution (declarative specs) solves that by centralizing the spec, making it versioned and composable, independent of any actual model. Engram layers just move the coordination problem earlier and lock it in. Coordination problems between models & providers would still…

Fair point. I don't know if it is cleaner or not.

The issue with a top-level spec, that I can see, is that models fall back to their training when it comes to tools. This is why I recommended the engram approach, because as far as I can tell the problem is a model problem not a systems problem.

Re: The M×N problem of tool calling and open-source models

#55

Post by R00mi »

hedgehog wrote: post #30
R00mi wrote: post #27

MCP is the wire format between agent and tool, not the format the model itself uses to emit the call. That part (Harmony, JSON, XML-ish) is still model-specific. So the M×N the article describes is really two problems stacked — MCP only solves the lower half. Also in practice Claude Code, Cursor and Codex handle the same MCP tool differently — required params, tool descriptions, response truncation. So MCP gives you…

But, like pancakes, usually the stack is described as building bottom-up. Can you relate the individual components to ingredients in a diner-style pancake breakfast?

Bottom-up preparation :)

- Plate — your product

- Pancake 1 — the model

- Syrup #1 — how the model emits the tool call (Harmony, JSON, XML-ish). Different flavor at every table.

- Pancake 2 — the client/agent

- Syrup #2 — MCP. Same brand at every table, finally.

- Pancake 3 — the MCP server (the tool)

Two syrups. MCP standardized the top one. The bottom one is still BYO at every table. That's the article. What do you think about that?

Re: The M×N problem of tool calling and open-source models

#56

Post by hedgehog »

R00mi wrote: post #55
hedgehog wrote: post #30

Earlier quoted context omitted.

But, like pancakes, usually the stack is described as building bottom-up. Can you relate the individual components to ingredients in a diner-style pancake breakfast?

Bottom-up preparation :) - Plate — your product - Pancake 1 — the model - Syrup #1 — how the model emits the tool call (Harmony, JSON, XML-ish). Different flavor at every table. - Pancake 2 — the client/agent - Syrup #2 — MCP. Same brand at every table, finally. - Pancake 3 — the MCP server (the tool) Two syrups. MCP standardized the top one. The bottom one is still BYO at every table. That's the article. What do you…

Please describe the process as a program written in standard Scheme.
Post reply on HN