31–40 of 55 posts
Earlier quoted context omitted.
Maybe the model is supposed to work in a customer support and needs access to Stripe to check payment details and hand out coupons for inconvenience?
I think you’d set the model up as you would any staff user of the platform - with authorised amounts it can issue without oversight and an escalation pathway if it needs more?
The "on by default" mitigation is mentioned at the very end: > Never enable "auto-confirm" on high-risk tools Maybe some tools should be able to specify to a client to never call it without a human approval. The security of the MCP ecosystem is basically based on human in the loop - otherwise things can go terribly wrong because of prompt injection and confused clients. And I'm not sure if current human approval sche…
This is just an ad for generalanalysis (itself an MCP tool).
OP is a 12-day old account that only posted about generalanalysis.
An LLM - which has functionally infinite unverifiable attack surface - directly wired into a payment system with high authentication. How could anyone anticipate this going wrong? I feel like everyone is saying 'we're still discovering what LLMs are good at' but it also feels like we really need to get in our collective conscious what they're really, really, bad at.
Somehow this site keeps making these posts and making it up front page and people keep sharing the same opinions
You sure? In their 5 month submit history, they’ve got one post with nearly 900 votes, this post, one post with 17, and a handful of others that didn’t break 10. Perhaps you’re confusing it with another site.
An LLM - which has functionally infinite unverifiable attack surface - directly wired into a payment system with high authentication. How could anyone anticipate this going wrong? I feel like everyone is saying 'we're still discovering what LLMs are good at' but it also feels like we really need to get in our collective conscious what they're really, really, bad at.
Companies are rushing or skipping a lot of required underlying security controls in a quest to be first or quick to market with what they think is transformative applications of AI. And so far, probably very few have gotten it right and generally only with serious spend. For instance, how many companies do you think have played with dedicated identities for each instance of their agents? Let alone hard-restricting th…
Earlier quoted context omitted.
>They are highlighting some pretty serious security flaws It's just a rehash of the same inherit flaw of LLMs.
that's reductive. this is effectively a disclosure. do you consider every disclosure write up an "ad" for the security researcher?
> 1 · Deploy an MCP Guard (three-command setup)
> A guardrail can help protect every tool call with a protective layer that blocks malicious or out-of-policy instructions in real time. Here is how to install the GA MCP guard which is open-source and requires no billing.
> $ pip install generalanalysis # install the guard
> $ ga login # browser-based auth
> $ ga configure
> MCP Guard protection enabled
Earlier quoted context omitted.
that's reductive. this is effectively a disclosure. do you consider every disclosure write up an "ad" for the security researcher?
I do if their "mitigation" looks like this: > 1 · Deploy an MCP Guard (three-command setup) > A guardrail can help protect every tool call with a protective layer that blocks malicious or out-of-policy instructions in real time. Here is how to install the GA MCP guard which is open-source and requires no billing. > $ pip install generalanalysis # install the guard > $ ga login # browser-based auth > $ ga configure >…
it's an evolving field. if anthropic doesn't have a solution should we just not do anything?
And here we are all over again. (double facepalm) I wouldn't touch MCP with a 100-foot pole.
An LLM - which has functionally infinite unverifiable attack surface - directly wired into a payment system with high authentication. How could anyone anticipate this going wrong? I feel like everyone is saying 'we're still discovering what LLMs are good at' but it also feels like we really need to get in our collective conscious what they're really, really, bad at.
If you didn’t catch it, this scenario was fabricated for this blog post. The company writing the post sells vulnerability testing tools.
This isn’t what a real production system even looks like. They’re using Claude Desktop. I mean I guess someone who doesn’t know better could connect Stripe and iMessage to Claude Desktop and then give the Stripe integration full permissions. It’s possible. But this post wasn’t an exploit of a real world system they found. They created it and then exploited it as an example. They sell services to supposedly scan for vulnerabilities like this.
An LLM - which has functionally infinite unverifiable attack surface - directly wired into a payment system with high authentication. How could anyone anticipate this going wrong? I feel like everyone is saying 'we're still discovering what LLMs are good at' but it also feels like we really need to get in our collective conscious what they're really, really, bad at.