Live data from Hacker News

Supabase MCP can leak your entire SQL database

generalanalysis.com

171–180 of 502 posts

Re: Supabase MCP can leak your entire SQL database

#171
post #156

Earlier quoted context omitted.

You said you wanted to take the one agent, split it into two agents, and add a third agent in between. It could be that we are equivocating on the currently-dubious definition of "agent" that has been being thrown around in the AI/LLM/MCP community ;P.

No, I didn't. An LLM context is just an array of strings. Every serious agent manages multiple contexts already.

If I have two agents and make them communicate, at what point should we start to consider them to have become a single agent?

Re: Supabase MCP can leak your entire SQL database

#172
post #61

Earlier quoted context omitted.

Pragmatically, does your responsible disclosure processes matter, when the resolution is “ask the LLM more times to not leak data, and add disclosures to the documentation”?

Absolutely astounding to me, having watched security culture evolve from "this will never happen", though "don't do that", to the modern world of multi-mode threat analysis and defense in depth... ...to see it all thrown in the trash as we're now exhorted, literally, to merely ask our software nicely not to have bugs.

[deleted]

Re: Supabase MCP can leak your entire SQL database

#173
post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

This "attack" can't be mitigated with prompting or guardrails though – the security needs to be implemented on the user level. The MCP server's db user should only have access to the tables and rows it's supposed to. LLMs simply can't be trusted to adhere to access policies, and any attempts to do that probably just limits the MCP server's capabilities without providing any real security.

Re: Supabase MCP can leak your entire SQL database

#174
post #93

Earlier quoted context omitted.

[3] https://supabase.com/.well-known/security.txt That "What we promise:" section reads like a not so subtle threat framing, rather than a collaborative, even welcoming tone one might expect. Signaling a legal risk which is conditionally withheld rather than focusing on, I don't know, trust and collaboration would deter me personally from reaching out since I have an allergy towards "silent threats". But, that's just…

I hadn't noticed it before, but it looks like that somewhat passive aggressive wording is a common phrase in responsible disclosure policies: https://www.google.com/search?q=%22If+you+have+followed+the+...

"Responsible disclosure policies" are mostly vendor exhortations to people who do a public service (finding vulnerabilities and publicly disclosing them) not to embarrass them too much. The fact they contain silly boilerplate is probably just a function of their overall silliness.

Re: Supabase MCP can leak your entire SQL database

#175
post #16

Earlier quoted context omitted.

There are no prepared statements for LLMs. It can't distinguish between your instructions and the data you provide it. So if you want the bot to be able to do certain actions, no prompt engineering can ever keep you safe. Of course, it probably shouldn't be connected and able to read random tables. But even if you want the bot to "only" be able to do stuff in the ticket system (for instance setting a priority) you're…

> It can't distinguish between your instructions and the data you provide it. Which is exactly why it is blowing my mind that anyone would connect user-generated data to their LLM that also touches their production databases.

>Which is exactly why it is blowing my mind that anyone would connect user-generated data to their LLM that also touches their production databases.

So many product managers are demanding this of their engineers right now. Across most industries and geographies.

Re: Supabase MCP can leak your entire SQL database

#176
post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

How does an e2e test for less capable LLMs look like, you call each LLM one by one? Aren't these tests flaky by the nature of LLMs, how do you deal with that?

Re: Supabase MCP can leak your entire SQL database

#178

CEO of General Analysis here (The company mentioned in this blogpost) First, I want to mention that this is a general issue with any MCPs. I think the fixes Supabase has suggested are not going to work. Their proposed fixes miss the point because effective security must live above the MCP layer, not inside it. The core issue that needs addressing here is distinguishing between data and instructions. A system needs to…

I broadly agree that "MCP-level" patches alone won't eliminate prompt-injection risk. Latest research also shows we can make real progress by enforcing security above the MCP layer, exactly as you suggest [1]. DeepMind's CaMeL architecture is a good reference model: it surrounds the LLM with a capability-based "sandbox" that (1) tracks the provenance of every value, and (2) blocks any tool call whose arguments origin…

> unless an explicit policy grants permission

Three months later, all devs have “Allow *” in their tool-name.conf

Re: Supabase MCP can leak your entire SQL database

#179
post #68

Earlier quoted context omitted.

Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked. It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with produc…

The main problem seems to me to be related to the ancient problem of escape sequences and that has never really been solved. Don't mix code (instructions) and data in a single stream. If you do sooner or later someone will find a way to make data look like code.

That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing.

What we call code, and what we call data, is just a question of convenience. For example, when editing or copying WMF files, it's convenient to think of them as data (mix of raster and vector graphics) - however, at least in the original implementation, what those files were was a list of API calls to Windows GDI module.

Or, more straightforwardly, a file with code for an interpreted language is data when you're writing it, but is code when you feed it to eval(). SQL injections and buffer overruns are a classic examples of what we thought was data being suddenly executed as code. And so on[0].

Most of the time, we roughly agree on the separation of what we treat as "data" and what we treat as "code"; we then end up building systems constrained in a way as to enforce the separation[1]. But it's always the case that this separation is artificial; it's an arbitrary set of constraints that make a system less general-purpose, and it only exists within domain of that system. Go one level of abstraction up, the distinction disappears.

There is no separation of code and data on the wire - everything is a stream of bytes. There isn't one in electronics either - everything is signals going down the wires.

Humans don't have this separation either. And systems designed to mimic human generality - such as LLMs - by their very nature also cannot have it. You can introduce such distinction (or "separate channels", which is the same thing), but that is a constraint that reduces generality.

Even worse, what people really want with LLMs isn't "separation of code vs. data" - what they want is for LLM to be able to divine which part of the input the user would have wanted - retroactively - to be treated as trusted. It's unsolvable in general, and in terms of humans, a solution would require superhuman intelligence.

--

[0] - One of these days I'll compile a list of go-to examples, so I don't have to think of them each time I write a comment like this. One example I still need to pick will be one that shows how "data" gradually becomes "code" with no obvious switch-over point. I'm sure everyone here can think of some.

[1] - The field of "langsec" can be described as a systematized approach of designing in a code/data separation, in a way that prevents accidental or malicious misinterpretation of one as the other.

Re: Supabase MCP can leak your entire SQL database

#180
one day everything private will be leaked and they'll blame it on misconfiguration by someone they can't even point a finger at. some contractor on another continent.

how many of you have auth/athr just one `if` away from disaster?

we will have a massive cloud leak before agi

Post reply on HN