21–30 of 55 posts
Every single one of these "vulnerabilities" is basically: - Set up a website without any input sanitization. - Hey look, you can take control of the database via SQL injection, therefore SQL is completely broken. - Here's a service you can use to prevent this at your company (which we happen to own).
An LLM - which has functionally infinite unverifiable attack surface - directly wired into a payment system with high authentication. How could anyone anticipate this going wrong? I feel like everyone is saying 'we're still discovering what LLMs are good at' but it also feels like we really need to get in our collective conscious what they're really, really, bad at.
For instance, how many companies do you think have played with dedicated identities for each instance of their agents? Let alone hard-restricting those identities (not via system prompts but with good old fashioned access controls) to only the data and functions they're supposed to be entitled to for just that session?
It's a pretty slim number. Only reason I'm not guessing zero is because it wouldn't surprise me if maybe one company got it right. But if there was a way to prove that nobody's doing this right, I'd bet money on it for laughs. These are things that in theory we should've been doing before AI happened, and yet it's all technical debt alongside every "low" or "medium" risk for most companies because up until now, no one could rationalize the spend.
Earlier quoted context omitted.
The setup itself is absurd. They gave their model full access to their Stripe account (including the ability to generate coupons of unlimited value) via MCP. The mitigation is - don't do that.
Maybe the model is supposed to work in a customer support and needs access to Stripe to check payment details and hand out coupons for inconvenience?
An ever increasing attack surface with each MCP connection.
N + 1 MCP connections + non-determinstic language model + sensitive data store = guaranteed disaster waiting to happen.
Earlier quoted context omitted.
The setup itself is absurd. They gave their model full access to their Stripe account (including the ability to generate coupons of unlimited value) via MCP. The mitigation is - don't do that.
Maybe the model is supposed to work in a customer support and needs access to Stripe to check payment details and hand out coupons for inconvenience?
Earlier quoted context omitted.
I don’t think that’s really fair. They are highlighting some pretty serious security flaws in MCP tools that are allowed to do some pretty privileged things. They don’t even mention their product till the very last section. Overall think it’s an excellent blog post.
>They are highlighting some pretty serious security flaws It's just a rehash of the same inherit flaw of LLMs.
> Never enable "auto-confirm" on high-risk tools
Maybe some tools should be able to specify to a client to never call it without a human approval.
The security of the MCP ecosystem is basically based on human in the loop - otherwise things can go terribly wrong because of prompt injection and confused clients.
And I'm not sure if current human approval scheme work, because the normalization of deviance is a real thing and humans don't like clicking "approve" all the time...