Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making MCP cheaper via CLI

kanyilmaz.me

Post reply on HN
Print view View original

61–70 of 127 posts

Re: Making MCP cheaper via CLI

#61

Post by sakesun »

_pdp_ wrote: post #23

There is some important context missing from the article. First, MCP tools are sent on every request. If you look at the notion MCP the search tool description is basically a mini tutorial. This is going right into the context window. Given that in most cases MCP tool loading is all or nothing (unless you pre-select the tools by some other means) MCP in general will bloat your context significantly. I think I counted…

From your description, GraphQL or SQL could be a good solution for AI context as well.

Re: Making MCP cheaper via CLI

#63

Post by foota »

Does tool calling in general bloat context, or is there something particular about MCP?

One thing I have read recently is that when you make a tool call it forces the model to go back to the agent. The effect of this is that the agent then has to make another request with all of the prompt (include past messages), these will be "cached" tokens, but they're still expensive. So if you can amortize the tool calls by having the model either do many at once or chaining them with something like bash you'll be better off.

I suspect this might be why cursor likes writing bash scripts so much, simple shell commands are going to be very token heavy because of the frequency of interrupts.

Re: Making MCP cheaper via CLI

#66

Post by charcircuit »

eggplantiny wrote: post #56

I'm looking at this from a slightly different level of abstraction. The CLI approach definitely has practical benefits for token reduction. Not stuffing the entire schema into the runtime context is a clear win. But my main interest lies less in "token cost" and more in "how we structure the semantic space." MCP is fundamentally a tool-level protocol. Existing paradigms like Skills already mitigate context bloat and…

>what if we normalize the semantic primitives above them (e.g., "search," "read," "create")?

Trying to dictate the abstractions that should be used is not bitter lesson pilled.

Re: Making MCP cheaper via CLI

#68

Post by cjonas »

sakesun wrote: post #61
_pdp_ wrote: post #23

There is some important context missing from the article. First, MCP tools are sent on every request. If you look at the notion MCP the search tool description is basically a mini tutorial. This is going right into the context window. Given that in most cases MCP tool loading is all or nothing (unless you pre-select the tools by some other means) MCP in general will bloat your context significantly. I think I counted…

From your description, GraphQL or SQL could be a good solution for AI context as well.

SQL is peak for data retrieval (obviously) but challenging to deploy for multitenant applications where you can't just give the user controlled agent a DB connection. I found it every effective to create a mini paquet "data ponds" on the fly in s3 and allow the agent to query it with duckdb (can be via tool call but better via a code interpreter). Nice thing with this approach is you can add data from any source and the agent can join efficiently.

Re: Making MCP cheaper via CLI

#69

Post by martinald »

cheriot wrote: post #49

Is there any redeeming quality of MCP vs a skill with CLI tool? Right now it looks like the latter is a clear winner. Maybe MCP can help segregate auto-approve vs ask more cleanly, but I don't actually see that being done.

MCP defines a consistent authentication protocol. This is the real issue with CLIs, each CLI can (and will) have a different way of handling authentication (env variables, config set, JSON, yml, etc).

But tbh there's no reason agents can't abstract this out. As long as a CLI has a --help or similar (which 99% do) with a description of how to login, then it can figure it out for you. This does take context and tool calls though so not hugely efficient.

Re: Making MCP cheaper via CLI

#70

Post by kanodiaayush »

If we use prompt caching - isn't a largish MCP tools section just like a fixed token penalty in return for higher speed at runtime, because tools don't need to be discovered on demand, and that's the better tradeoff? At least for the most powerful models it doesn't feel like their quality goes down much with a few MCP servers. I might be missing something.
Post reply on HN