Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stateless MCP has recaptured my interest

simonwillison.net

Post reply on HN
Print view View original

201–210 of 246 posts

Re: Stateless MCP has recaptured my interest

#201

Post by simonw »

troupo wrote: post #137
ricardobeat wrote: post #116

Earlier quoted context omitted.

Skills don’t need to be preloaded, can be invoked by name on demand; and when you list available skills, a one-line description for each is enough.

And all those "names and one-line descriptions" live in an .md file and of course pollute your context. And then the model will randomly decide to load the .md file that is the "skill" randomly at any point. Again polluting the context. There's no magic.

I've seen implementations of a skill search, where instead of loading all descriptions into the initial prompt there's a tool the agent can call to search available skills and see if one might match their new task.

Re: Stateless MCP has recaptured my interest

#202

Post by NiloCK »

CrompyBlompers wrote: post #194
troupo wrote: post #137

Earlier quoted context omitted.

And all those "names and one-line descriptions" live in an .md file and of course pollute your context. And then the model will randomly decide to load the .md file that is the "skill" randomly at any point. Again polluting the context. There's no magic.

Most people seem to mean "Skills don't pollute the context window [with their entire contents]", they just elide the part in brackets. There is no magic and people should be more precise when discussing skills.

Yes, but I expect that the elided part is less important than people assume it is.

Token count is a less important factor in context pollution than idea count. The worst of the rot factors are when models latch onto irrelevant information, or over-index on some vague idea/suggestion as if it was a hard direction, and then go off course.

The names + one-line descriptions of 10 tools can do as much (or more!) to distract the focus and intentionality of an agent than a 30k token exhaustive API documentation of some tool.

Re: Stateless MCP has recaptured my interest

#203

Post by troupo »

simonw wrote: post #201
troupo wrote: post #137

Earlier quoted context omitted.

And all those "names and one-line descriptions" live in an .md file and of course pollute your context. And then the model will randomly decide to load the .md file that is the "skill" randomly at any point. Again polluting the context. There's no magic.

I've seen implementations of a skill search, where instead of loading all descriptions into the initial prompt there's a tool the agent can call to search available skills and see if one might match their new task.

Search itself pollutes the context. With the need to maintain the tools, the search, the search results etc. in context. And wasting tokens while interpreting results.

Re: Stateless MCP has recaptured my interest

#204

Post by drdexebtjl »

heckintime wrote: post #200
drdexebtjl wrote: post #30

Earlier quoted context omitted.

Why can’t the LLM you’re talking to on the web have access to your CLI tools? When you talk to an LLM on the web, the harnesses spin up a fresh environment (I would hope it’s a VM…) so that the LLM can do stuff like run arbitrary Python and Bash scripts to complete the task you asked it for. There’s no reason why you shouldn’t be able to customize this environment to add whatever CLI tools and credentials you need fo…

Could be more expensive to provide a computing environment.

It can’t possibly be more expensive than doing equivalent work using context instead of RAM and inference instead of CPU.

Unless we’ve got the wrong balance of compute availability and inference availability right now, but I would expect the market to stabilize at some point.

Re: Stateless MCP has recaptured my interest

#205

Post by ljm »

lbourdages wrote: post #188
blitzar wrote: post #148

Earlier quoted context omitted.

that would be tokens, vibes and guesses all the way down

As is the entirety of using an LLM? If you want something deterministic and efficient you get the agent to write a script, verify it, then run it yourself.

Bring back FastCGI, enable the setting in Apache to list a directory, and point the agent to your cgi-bin directory full of all your tools. Calling the bin without any input provides the help text or the interface to the tool.

You can probably throw together a passible auth system on top of it that works with the htaccess format. Which is another cgi script but chowned so it can't be called from the outside. If not that, public key crypto - give your agents certificates that are part of the chain of trust. Mint those out of band and give them an insanely short TTL. Fail2ban or something like that to prevent abuse (agent getting handsy with the server gets blocked, credentials revoked).

The side benefit is that everybody starts writing perl and awk again. Who needs a JS runtime when you have CGI?

Re: Stateless MCP has recaptured my interest

#206

Post by jimbokun »

lexicality wrote: post #105
ai_critic wrote: post #50

It is amazing that folks getting paid several hundred thousand dollars a year...reinvented RPC-over-HTTP/JSON. You too, fellow web developers, are smart enough to work at Anthropic. I would love to see a proper engineering post-mortem for how this happened.

They didn't reinvent it, it's literally using JSONRPC. The thing they "invented" was coming up with a loose standard for how to implement your RPC endpoints in a way that the AI was pre-trained to interact with. The actual implementation may have been very janky but the idea itself is sound: If you want to interact with arbitrary services then providing one single protocol (ideally not JSONRPC but hey) with a standar…

> was coming up with a loose standard for how to implement your RPC endpoints in a way that the AI was pre-trained to interact with.

Why not just pre-train the model to interact with JSONRPC endpoints?

Re: Stateless MCP has recaptured my interest

#207

Post by miroljub »

mi_lk wrote: post #181
Eldodi wrote: post #87

Earlier quoted context omitted.

MCP context bloat is a solved problem since at least February. Both OpenAI and Anthropic support Client-side MCP tool search, which makes MCP context loading as efficient as the Skills one (progressive discovery). Code mode is great but not needed anymore in 95% of use-cases.

Interesting, then I wonder if Pi having Skills support but not MCP support by default still makes sense

Pi does support MCP through an excellent pi-mcp-adapter plugin.

Re: Stateless MCP has recaptured my interest

#208

Post by jimbokun »

It seems to me an alternative to MCP would be a shell implemented to only allow access to white listed commands.

I also feel like a lot of knowledge about how to lock down a multi-user system has been lost. Back when many people shared a single computer through terminals, admins had a lot of responsibility to create a permissions structure such that one user couldn't access another's data or hog too many resources that would impact other users on the system. We are running computers with the similar permission models (Mac and Linux are Unix, Windows has similar controls) but we can't manage to lock down shell access to trust an LLM with it as if it were a human user we don't fully trust.

Re: Stateless MCP has recaptured my interest

#210

Post by tyree731 »

jimbokun wrote: post #206
lexicality wrote: post #105

Earlier quoted context omitted.

They didn't reinvent it, it's literally using JSONRPC. The thing they "invented" was coming up with a loose standard for how to implement your RPC endpoints in a way that the AI was pre-trained to interact with. The actual implementation may have been very janky but the idea itself is sound: If you want to interact with arbitrary services then providing one single protocol (ideally not JSONRPC but hey) with a standar…

> was coming up with a loose standard for how to implement your RPC endpoints in a way that the AI was pre-trained to interact with. Why not just pre-train the model to interact with JSONRPC endpoints?

There's more to it than that. MCP adds the concept of the API endpoints describing themselves (giving agents context), standardizes how auth works, and so on. This could all be done by packaging existing standards, but MCP in many ways is packaging and organizing existing standards, minus the previous stateful-first design.

I do think a lot of the standard would have been easier if they'd been stateless first, and allowed for GET requests, but it evolved from a starting point that has since evolved.

Post reply on HN