If you give your service role key to an LLM and then bad shit happens you have only yourself to blame.
Supabase MCP can leak your entire SQL database
221–230 of 502 posts
Re: Supabase MCP can leak your entire SQL database
#222Earlier quoted context omitted.
You can have the MCP talking directly to the database if you want! You just can't have it in this configuration of a single context that both has all the tool calls and direct access to untrusted data.
Whichever model/agent is coordinating between other agents/contexts can itself be corrupted to behave unexpectedly. Any model in the chain can be. The only reasonable safeguard is to firewall your data from models via something like permissions/APIs/etc.
If you're just speaking in the abstract, all code has bugs, and some subset of those bugs will be security vulnerabilities. My point is that it won't have this bug.
Re: Supabase MCP can leak your entire SQL database
#223Earlier quoted context omitted.
Seems they can't imagine the constraints being implemented as code a human wrote so they're just imagining you're adding another LLM to try to enforce them?
(EDIT: THIS WAS WRONG.) [[FWIW, I definitely can imagine that (and even described multiple ways of doing that in a lightweight manner: pattern whitelisting and fine-grained permissions); but, that isn't what everyone has been calling an "agent" (aka, an LLM that is able to autonomously use tools, usually, as of recent, via MCP)? My best guess is that the use of "agent code" didn't mean the same version of "agent" tha…
It's fine if you want to talk about other bugs that can exist; I'm not litigating that. I'm talking about foreclosing on this bug.
Re: Supabase MCP can leak your entire SQL database
#224Earlier quoted context omitted.
No, I didn't. An LLM context is just an array of strings. Every serious agent manages multiple contexts already.
FWIW, I don't think you can enforce that correctly with human code either, not "in between those contexts"... what are you going to filter/interpret? If there is any ability at all for arbitrary text to get from the one LLM to the other, then you will fail to prevent the SQL-capable LLM from being attacked; and like, if there isn't, then is the "invariant" you are "enforcing" that the one LLM is only able to communic…
There doesn't have to be an ability for "arbitrary text" to go from one context to another. The first context can produce JSON output; the agent can parse it (rejecting it if it doesn't parse), do a quick semantic evaluation ("which tables is this referring to"), and pass the structured JSON on.
I think at some point we're just going to have to build a model of this application and have you try to defeat it.
Re: Supabase MCP can leak your entire SQL database
#225Earlier quoted context omitted.
Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked. It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with produc…
No it can't work. Not in general. And MCP is "in general". Whereas custom coded tool use might be secure on a case by case basis if the coder knows what they are doing.
MCP is a red herring here.
Re: Supabase MCP can leak your entire SQL database
#226Earlier quoted context omitted.
The analogy I like is... humans[0]. There's literally no way to separate "code" and "data" for humans. No matter how you set things up, there's always a chance of some contextual override that will make them reinterpret the inputs given new information. Imagine you get a stack of printouts with some numbers or code, and are tasked with typing them into a spreadsheet. You're told this is all just random test data, but…
> There's literally no way to separate "code" and "data" for humans It's basically phishing with LLMs, isn't it?
Yes.
I've been saying it ever since 'simonw coined the term "prompt injection" - prompt injection attacks are the LLM equivalent of social engineering, and the two are fundamentally the same thing.
Re: Supabase MCP can leak your entire SQL database
#227Earlier quoted context omitted.
If I have two agents and make them communicate, at what point should we start to consider them to have become a single agent?
They don’t communicate directly. They’re mediated by agent code.
Now I'm more confused. So does that mediating agent code constitute a separate agent Z, making it three agents X,Y,Z? Explicitly or not (is this the meaningful distinction?) information flowing between them constitutes communication for this purpose.
It's a hypothetical example where I already have two agents and then make one affect the other.
Re: Supabase MCP can leak your entire SQL database
#228Earlier quoted context omitted.
It's an MCP for your database, ofcourse it's going to execute SQL. It's your responsibility for who/what can access the MCP that you've pointed at your database.
Except without any authentication and authorization layer. Remember, the S in MCP is for "security." Also, you can totally have an MCP for a database that doesn't provide any SQL functionality. It might not be as flexible or useful, but you can still constrain it by design.
No part of what happened in this bug report has anything to do with authentication and authorization. These developers are using the MCP equivalent of a `psql` prompt. They assume full access.
I think this "S in MCP" stuff is a really handy indicator for when people have missed the underlying security issue, and substituted some superficial thing instead.
Re: Supabase MCP can leak your entire SQL database
#229Earlier quoted context omitted.
> Adding more agents is still just mitigating the issue This is a big part of how we solve these issues with humans https://csrc.nist.gov/glossary/term/Separation_of_Duty https://en.wikipedia.org/wiki/Separation_of_duties https://en.wikipedia.org/wiki/Two-person_rule
The difference between humans and LLM systems is that, if you try 1,000 different variations of an attack on a pair of humans, they notice. There are plenty of AI-layer-that-detects-attack mechanisms that will get you to a 99% success rate at preventing attacks. In application security, 99% is a failing grade. Imagine if we prevented SQL injection with approaches that didn't catch 1% of potential attacks!
That's a wrong approach.
You can't have 100% security when you add LLMs into the loop, for the exact same reason as when you involve humans. Therefore, you should only include LLMs - or humans - in systems where less than 100% success rate is acceptable, and then stack as many mitigations as it takes (and you can afford) to make the failure rate tolerable.
(And, despite what some naive takes on infosec would have us believe, less than 100% security is perfectly acceptable almost everywhere, because that's how it is for everything except computers, and we've learned to deal with it.)
Re: Supabase MCP can leak your entire SQL database
#230Earlier quoted context omitted.
The difference between humans and LLM systems is that, if you try 1,000 different variations of an attack on a pair of humans, they notice. There are plenty of AI-layer-that-detects-attack mechanisms that will get you to a 99% success rate at preventing attacks. In application security, 99% is a failing grade. Imagine if we prevented SQL injection with approaches that didn't catch 1% of potential attacks!
That's a wrong approach. You can't have 100% security when you add LLMs into the loop, for the exact same reason as when you involve humans. Therefore, you should only include LLMs - or humans - in systems where less than 100% success rate is acceptable, and then stack as many mitigations as it takes (and you can afford) to make the failure rate tolerable. (And, despite what some naive takes on infosec would have us…
Sure you can. You just design the system to assume the LLM output isn't predictable, come up with invariants you can reason with, and drop all the outputs that don't fit the invariants. You accept up front the idea that a significant chunk of benign outputs will be lossily filtered in order to maintain those invariants. This just isn't that complicated; people are super hung up on the idea that an LLM agent is a loop around a single "LLM session", which is not how real agents work.