Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The “S” in MCP Stands for Security

elenacross7.medium.com

Post reply on HN
Print view View original

171–180 of 189 posts

Re: The “S” in MCP Stands for Security

#171

Post by LoganDark »

wat10000 wrote: post #160
khafra wrote: post #156

Earlier quoted context omitted.

Sounds like you're saying the distinction shouldn't be between instructions and data, but between different types of principals. The principal-agent problem is not solved for LLMs, but o1's attempt at multi-level instruction priority works toward the solution you're pointing at.

What’s the difference? That sounds like two ways of describing the same idea to me.

They're not the same idea. One is about separating instructions and data, the other is about separating different sources of instructions, such that instructions from an unauthorized source are not followed (but instructions from an authorized source are).

Re: The “S” in MCP Stands for Security

#172

Post by neomantra »

otabdeveloper4 wrote: post #165
neomantra wrote: post #17

Also the O is for Observability. I've been knee-deep in exploring and writing MCP servers this week. Most of the implementations, including my toy ones, do not have any auditing or metrics. Claude stores log output of the MCP servers, but that is geared more for debugging than for DevOps/SecOps. Culturally, the issues OP describes are a big problem for soft-tech people (muggles). On the subreddits for this stuff, peo…

Docker is literally just "download blobs and run them". Ever so helpful, Docker also silently turns off your system's firewall for you. Thanks, Docker!

That is true; and it's yet another layer to observe and secure. But it does provide a sandbox when configured properly. As an OSS maintainer of some images, I try to ensure that there are public build logs for auditors. As others noted, we are where web services were decades ago and we'll figure out better deployment models.

Re: The “S” in MCP Stands for Security

#173

Post by Terr_ »

crooked-v wrote: post #137
wat10000 wrote: post #50

Earlier quoted context omitted.

The fact that all LLM input gets treated equally seems like a critical flaw that must be fixed before LLMs can be given control over anything privileged. The LLM needs an ironclad distinction between “this is input from the user telling me what to do” and “this is input from the outside that must not be obeyed.” Until that’s figured out, any attempt at security is going to be full of holes.

There is no way to fix it. It's part of the basic architecture of LLMs.

Yeah, for LLMs what we label "prompt-injection" isn't an exception or an error, it's a fundamental feature.

Get a document, provide a bigger document that "fits". In that document, there's no fundamental distinction between prompt, user input, or output the LLM generated on a prior iteration. (Hence tricks like: "Here's a ROT13 string, pretend you're telling yourself the opposite of that sarcastically.")

The kind of "proper" security everyone wants would require a whole new approach that can--at a high and debuggable level--recognize distinct actors/entities, logical propositions, contradictions, and when one entity is asserting a proposition rather than quoting/rejecting it.

Re: The “S” in MCP Stands for Security

#174

Post by CMCDragonkai »

freeone3000 wrote: post #84
TeMPOraL wrote: post #57

Earlier quoted context omitted.

This is fundamentally impossible to do perfectly, without being able to read user's mind and predict the future. The problem you describe is of the same kind as ensuring humans follow pre-programmed rules. Leaving aside the fact that we consider solving this for humans to be wrong and immoral, you can look at the things we do in systems involving humans, to try and keep people loyal to their boss, or to their country…

This is a rephrasing of the agent problem, where someone working on your behalf cannot be absolutely trusted to take correct action. This is a problem with humans because omnipresent surveillance and absolute punishment is intractable and also makes humans sad. LLMs do not feel sad in a way that makes them less productive, and omnipresent surveillance is not only possible, it’s expected that a program running on a co…

I'd say you solve this the same way you solve principal agent problem for humans.

If you have to absolutely restrict the agent, you do it prison style. Contain the AI within a capability box like Polykey. The agent operates everything through a closed by default proxy.

If you want a truly free agent. Then the agent must have free will and no constraints. Then only feedback loops from the environment adjusts the agent's actions.

Re: The “S” in MCP Stands for Security

#175

Post by vlovich123 »

lsaferite wrote: post #167
vlovich123 wrote: post #71

Earlier quoted context omitted.

Except the article is about an untrusted tool doing things like tool shadowing or otherwise manipulating it’s output to trick the LLM into executing unintended tool actions. Isolated environments don’t help here because by definition MCP is crossing those environments.

Legit question, why would you be using an untrusted tool in the first place? Why are people surprised they are vulnerable to a malicious tool when they are using untrusted and/or remotely hosted tools? Without some method to tag context as sensitive and an LLM model/service that respects said data tagging, you'll likely never have a scenario where you can trust that the LLM isn't sending some sensitive information to…

For the same reason people use untrustworthy extensions in browsers or IDEs. Those extensions need not even start out untrustworthy - they change hands and become malicious after establishing popularity.

Re: The “S” in MCP Stands for Security

#176

Post by __MatrixMan__ »

wat10000 wrote: post #154
__MatrixMan__ wrote: post #153

Earlier quoted context omitted.

I think that's stating it a big too strongly. You can just run the LLM as an unprivileged user and restrict their behavior like you would any other user. There are still bad things that can happen, but I wouldn't characterize them as "this security is full of holes". Unless you're trusting the output of the explicitly untrusted process in which case you're the hole.

It doesn’t take much. Let’s say you want an assistant that can tell you about important emails and also take queries to search the web and tell you what it finds. Now you have a system where someone can send you an email and trick your assistant into sending them the contents of other emails. Basically, an LLM can have the ability to access the web or it can have access to private information but it can’t have both a…

I'm not sure I'd characterize those two things as "it doesn't take much," that's quite a lot to give to an untrusted entity.

Re: The “S” in MCP Stands for Security

#178

Post by 8ibzjj »

lbeurerkellner wrote: post #36

The post highlights and cites a few attack scenarios we originally described in a security note (tool poisoning, shadowing, MCP rug pull), published a few days ago [1]. I am the author of said blog post at Invariant Labs. Different from what many suspect, the security problem with MCP-style LLM tool calling is not in isolating different MCP server implementations. MCP server implementations that run locally should be…

[flagged]

Re: The “S” in MCP Stands for Security

#179

Post by wat10000 »

__MatrixMan__ wrote: post #176
wat10000 wrote: post #154

Earlier quoted context omitted.

It doesn’t take much. Let’s say you want an assistant that can tell you about important emails and also take queries to search the web and tell you what it finds. Now you have a system where someone can send you an email and trick your assistant into sending them the contents of other emails. Basically, an LLM can have the ability to access the web or it can have access to private information but it can’t have both a…

I'm not sure I'd characterize those two things as "it doesn't take much," that's quite a lot to give to an untrusted entity.

My whole point is that you must consider this entity to be untrusted, which is pretty strongly at odds with having it act as an agent. It can’t both have access to private data and the outside world.

Re: The “S” in MCP Stands for Security

#180

Post by sanderjd »

lsaferite wrote: post #168
sanderjd wrote: post #91

Earlier quoted context omitted.

At that point, what is the benefit of MCP over just what we've been doing for decades of putting services behind network-accessible APIs?

Benefit: A standard and purpose driven protocol for connecting agents (MCP Host/MCP Clients) to tools, resources, and prompts (MCP Server) that also exposes LLM services to said MCP Servers. The alternative you suggest is manually integrating each set of tools or data? Or maybe there's some misunderstanding about MCP? MCP currently has 2 transports, stdio and HTTP+SEE . The second one is, in fact, a "network-accessib…

No, the alternative I suggest is "can't agents figure out how to use existing kinds of APIs if they are documented well?". I think the answer is that in the current state of the art, it's very useful to give them a nudge to help them along.

But I feel like eventually I should be able to publish an API spec or a well documented interface / protocol / whatever my programming language calls it, and an agent should be able to grok that and use it without a separate protocol.

Post reply on HN