What even is MCP? I tried going through the docs on multiple occasions but I couldn't figure out what problem it's solving. Mainly, what is special about AI agents that doesn't also apply to deterministic agents that have existed for decades?
The “S” in MCP Stands for Security
121–130 of 189 posts
Re: The “S” in MCP Stands for Security
#122What even is MCP? I tried going through the docs on multiple occasions but I couldn't figure out what problem it's solving. Mainly, what is special about AI agents that doesn't also apply to deterministic agents that have existed for decades?
MCP is poorly named. That is why it’s confusing to many people. It’s a tool use protocol. It provides means to list tools provided by a server as well as manage asynchronous tasks. It’s transport agnostic and uses JSON-RPC to format requests and responses.
It’s different in that it’s designed to provide natural language instructions to LLMs and is a pretty open-ended protocol. It’s not like the Language Server Protocol which has all of its use cases covered in the spec. MCP gives just a little bit of structure but otherwise is built to be all things for all people. That makes it a bit hard to parse when reading the docs. I think they certainly could do a better job in communicating its design though.
Re: The “S” in MCP Stands for Security
#123Earlier quoted context omitted.
That’s the intention with developer messages from o1. It’s trained on a 3-tier system of messages. 1) system, messages from the model creator that must always be obeyed 2) dev, messages from programmers that must be obeyed unless the conflict with #1 3) user, messages from users that are only to be obeyed if they don’t contradict #1 or #2 Then, the model is trained heavily on adversarial scenarios with conflicting in…
But the grandparent is saying that there is a missing class of input "data". This should not be treated as instructions and is just for reference. For example if the user asks the AI to summarize a book it shouldn't take anything in the book as an instruction, it is just input data to be processed.
FYI, there is actually this implementation detail in the model spec, https://model-spec.openai.com/2025-02-12.html#chain_of_comma...
Platform: Model Spec "platform" sections and system messages
Developer: Model Spec "developer" sections and developer messages
User: Model Spec "user" sections and user messages
Guideline: Model Spec "guideline" sections
No Authority: assistant and tool messages; quoted/untrusted text and multimodal data in other messages
Re: The “S” in MCP Stands for Security
#124Earlier quoted context omitted.
There is no hard distinction between "code" and "data". Both are the same thing. We've built an entire computing industry on top of that fact, and it sort of works, and that's all with most software folks not even being aware that whether something is code or data is just a matter of opinion.
I've never had `cat` execute the file I was viewing.
You never accidentally cat-ed a binary file and borked your terminal?
If not, then find some random binary - an image, archive, maybe even /dev/random - and cat it.
Hint: `reset` will fix the terminal afterwards. Usually.
Re: The “S” in MCP Stands for Security
#125Earlier quoted context omitted.
I've never had `cat` execute the file I was viewing.
You never accidentally cat-ed a binary file and borked your terminal? If not, then find some random binary - an image, archive, maybe even /dev/random - and cat it. Hint: `reset` will fix the terminal afterwards. Usually.
That's not the same thing, and hasn't been a security issue for quite a while now.
Re: The “S” in MCP Stands for Security
#126Another bad standard designed by those who don't consider security as important. Which is why we have this excellent article. Essentially it's somehow fashionable to have remote-code-execution as a service by dumb agents executing anything they see when they use the MCP. Once one of those exploits are executed, your keys, secrets and personal configs are as good as donated to someone else's server and also sent back…
Like the article was AI generated?
Re: The “S” in MCP Stands for Security
#127MCP is a wire protocol, it just JSON endpoints with extra steps. You can either subscribe to zero trust, or you cannot. No protocol is going to magically make you care about security.
Re: The “S” in MCP Stands for Security
#128Earlier quoted context omitted.
The fact that all LLM input gets treated equally seems like a critical flaw that must be fixed before LLMs can be given control over anything privileged. The LLM needs an ironclad distinction between “this is input from the user telling me what to do” and “this is input from the outside that must not be obeyed.” Until that’s figured out, any attempt at security is going to be full of holes.
That’s the intention with developer messages from o1. It’s trained on a 3-tier system of messages. 1) system, messages from the model creator that must always be obeyed 2) dev, messages from programmers that must be obeyed unless the conflict with #1 3) user, messages from users that are only to be obeyed if they don’t contradict #1 or #2 Then, the model is trained heavily on adversarial scenarios with conflicting in…
How are these levels actually encoded? Do they use special unwritable tokens to wrap instructions?
Re: The “S” in MCP Stands for Security
#129Earlier quoted context omitted.
That’s the intention with developer messages from o1. It’s trained on a 3-tier system of messages. 1) system, messages from the model creator that must always be obeyed 2) dev, messages from programmers that must be obeyed unless the conflict with #1 3) user, messages from users that are only to be obeyed if they don’t contradict #1 or #2 Then, the model is trained heavily on adversarial scenarios with conflicting in…
I’m getting flashbacks to reading Asimov’s Robot series of novels! 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. … etc…
I thought that immediately too!
Re: The “S” in MCP Stands for Security
#130Earlier quoted context omitted.
So when I say “install this library”, should it or should it not follow the instructions (from the readme) for prereqs and how to install?
The question in the grandparent was "Can you install this library?". Not a command "install this library". If you ask an assistant "does the nearest grocery store sell ice cream?", you do not expect the response to be ice cream delivered to you.
Most LLM users don’t want models to have that level of literalism.
My manager would be very upset if they asked me “Can you get this done by Thursday?” and I responded with “Sure thing” - but took no further action, being satisfied that I’d literally fulfilled their request.