Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The “S” in MCP Stands for Security

elenacross7.medium.com

Post reply on HN
Print view View original

141–150 of 189 posts

Re: The “S” in MCP Stands for Security

#141

Post by fsndz »

ryoshu wrote: post #99
fsndz wrote: post #95

Earlier quoted context omitted.

So why are people so excited about MCP, and so suddenly? I think you know the answer by now: hype. Mostly hype, with a bit of the classic fascination among software engineers for architecture. You just say Model Context Protocol, server, client, and software engineers get excited because it’s a new approach — it sounds fancy, it sounds serious. https://www.lycee.ai/blog/why-mcp-is-mostly-bullshit

“For every complex problem there is a solution which is clear, simple and wrong.”—HL Mencken

this is top notch commentary

Re: The “S” in MCP Stands for Security

#142

Post by yencabulator »

spacebanana7 wrote: post #130
yencabulator wrote: post #108

Earlier quoted context omitted.

The question in the grandparent was "Can you install this library?". Not a command "install this library". If you ask an assistant "does the nearest grocery store sell ice cream?", you do not expect the response to be ice cream delivered to you.

Most LLM users don’t want models to have that level of literalism. My manager would be very upset if they asked me “Can you get this done by Thursday?” and I responded with “Sure thing” - but took no further action, being satisfied that I’d literally fulfilled their request.

Sure, that particular prompt is ambiguous. Feel free to imagine it to be more of an informational question, even one asking for just yes/no.

However, when people are talking about the "critical flaw" in LLMs, of which this "tool shadowing" attack is an example of, they're talking about how the LLMs cannot differentiate between text that is supposed to give them instructions and text that is supposed to be just for reference.

Concretely, today, ask an LLM "when was Elvis born", something in your MCP stack might be poisoning the LLM content window and causing another MCP tool to leak your SSH keys. I don't think you can argue that the user intended for that.

Re: The “S” in MCP Stands for Security

#143

Post by yencabulator »

TeMPOraL wrote: post #131
yencabulator wrote: post #125

Earlier quoted context omitted.

That's not the same thing, and hasn't been a security issue for quite a while now.

It is the same thing, that's the point. It all depends on how you look at it. Most software is trying to enforce a distinction between "code" and "data", in the sense that whatever we call "data" can only cause very limited set of things to happen - but that's just the program rules that make this distinction, fundamentally it doesn't exist. And thus, all it takes is some little bug in your input parser, or in whatev…

Yes, sure. In a normal computer, the differentiation between data and executable is done by the program being run. Humans writing those programs naturally can make mistakes.

However, the rules are being interpreted programmatically, deterministically. It is possible to get them right, and modern tooling (MMUs, operating systems, memory-safe programming languages, etc) is quite good at making that boundary solid. If this wasn't utterly, overwhelmingly, true, nobody would use online banking.

With LLMs, that boundary is now just a statistical likelihood. This is the problem.

Re: The “S” in MCP Stands for Security

#144

Post by daxfohl »

teaearlgraycold wrote: post #122
daxfohl wrote: post #121

What even is MCP? I tried going through the docs on multiple occasions but I couldn't figure out what problem it's solving. Mainly, what is special about AI agents that doesn't also apply to deterministic agents that have existed for decades?

MCP is poorly named. That is why it’s confusing to many people. It’s a tool use protocol. It provides means to list tools provided by a server as well as manage asynchronous tasks. It’s transport agnostic and uses JSON-RPC to format requests and responses. It’s different in that it’s designed to provide natural language instructions to LLMs and is a pretty open-ended protocol. It’s not like the Language Server Protoc…

Yeah, some more concrete examples would help. LSP docs make a lot more sense in that they lay out the problems that it solves: the many-to-many issue and the redundant-implementations-of-parsers-for-a-language issue. Maybe the USB(-C?) comparison is more apt, though I imagine most software engineers know less about that one. And IIUC the "-C" is just a physical component and not part of the protocol(?)

Anyway, sounds like we'll see a v2 and v3 and such of the protocol before long, to deal with some of the issues in the article.

Re: The “S” in MCP Stands for Security

#145

Post by a_vanderbilt »

jiggawatts wrote: post #116
rybosome wrote: post #80

Earlier quoted context omitted.

That’s the intention with developer messages from o1. It’s trained on a 3-tier system of messages. 1) system, messages from the model creator that must always be obeyed 2) dev, messages from programmers that must be obeyed unless the conflict with #1 3) user, messages from users that are only to be obeyed if they don’t contradict #1 or #2 Then, the model is trained heavily on adversarial scenarios with conflicting in…

I’m getting flashbacks to reading Asimov’s Robot series of novels! 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. … etc…

Asimov had a penchant for predicting the future, and it's been fascinating seeing aspects of his vision in "I, Robot" come to pass.

Re: The “S” in MCP Stands for Security

#146

Post by never_inline »

defjosiah wrote: post #123
kevincox wrote: post #98

Earlier quoted context omitted.

But the grandparent is saying that there is a missing class of input "data". This should not be treated as instructions and is just for reference. For example if the user asks the AI to summarize a book it shouldn't take anything in the book as an instruction, it is just input data to be processed.

FYI, there is actually this implementation detail in the model spec, https://model-spec.openai.com/2025-02-12.html#chain_of_comma... Platform: Model Spec "platform" sections and system messages Developer: Model Spec "developer" sections and developer messages User: Model Spec "user" sections and user messages Guideline: Model Spec "guideline" sections No Authority: assistant and tool messages; quoted/untrusted text a…

This still does not seem to fix the OP vulnerability? All tool call specs will be at same privilege level.

Re: The “S” in MCP Stands for Security

#147

Post by never_inline »

turnsout wrote: post #97
turnsout wrote: post #45

Earlier quoted context omitted.

Here's the better design: have agents communicate via Mastodon. Take a basic JSON payload, encrypt it using basic public key encryption, and attach it to a DM. This is far better than designing an entirely new protocol, as ActivityPub and Mastodon already have everything you need, including an API. Now, that's just transport security. If you expose a server that will execute arbitrary commands, nothing can protect yo…

If you're downvoting, can you explain why you disagree?

Because it's not an encryption problem. It's a "you can override instructions from other servers" problem.

Re: The “S” in MCP Stands for Security

#148

Post by never_inline »

TeMPOraL wrote: post #32

Here's a challenge: sketch a better design, that: 1. Is properly secure, to whatever standards will stop people writing "S Stands for Security" articles, and 2. Allows programs implementing it to provide the same set of features the most useful MCPs do now, without turning automatic functionality into one requiring manual user confirmations, and generally without defeating the purpose of the entire idea, and 3. Doesn…

A. Implement guardrails (like already done against prompt injection).

Invariant blog post mentions this:

> Conclusion: Agents require extensive, highly-contextual guardrailing and security solutions

> As one of our core missions at Invariant, we absolutely cannot stress enough how important it is to rely on extensive guardrailing with AI models and their actions. We come to this conclusion repeatedly, as part of our research and engineering work on agentic systems. The MCP ecosystem is no exception to this rule. Security must be implemented end-to-end, including not only the tool descriptions but also the data that is being passed to and from the AI model.

B. Version the tool descriptions so that they can be pinned and do not change (same way we do for libraries and APIs).

C. Maybe in future, LLMs can implement some sort of "instruction namespacing" - where the developer would be able to say any instruction in this prompt is only applicable when doing X, Y, Z.

Re: The “S” in MCP Stands for Security

#150

Post by AlexCoventry »

sanderjd wrote: post #91
AlexCoventry wrote: post #59

Earlier quoted context omitted.

It seems to me that the solution is to run this stuff in a securely isolated environment such as a VM, dedicated machine, or VPC, where you don't care about the secrets it has access to, and don't really care about corruption of the data in the environment. Then you have to carefully audit any products you take from that environment, if you want to run them in a more sensitive context. I don't think this is really an…

At that point, what is the benefit of MCP over just what we've been doing for decades of putting services behind network-accessible APIs?

Having a robot perform increasingly sophisticated tasks in your development environment still seems like a win in certain circumstances.
Post reply on HN