Live data from Hacker News

Supabase MCP can leak your entire SQL database

generalanalysis.com

261–270 of 502 posts

Re: Supabase MCP can leak your entire SQL database

#261
post #198

Earlier quoted context omitted.

That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing. What we call code, and what we call data, is just a question of convenience. For example, when editing or copying WMF files, it's convenient to think of them as data (mix of raster and vector graphics) - however, at least in the original implementation,…

> That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing. Sorry to perhaps diverge into looser analogy from your excellent, focused technical unpacking of that statement, but I think another potentially interesting thread of it would be the proof of Godel’s Incompleteness Theorem, in as much as the Godel Sent…

Never thought of this before, despite having read multiple books on godel and his first theorem. But I think you’re absolutely right - that a whole class of code injection attacks are variations of the liars paradox.

Re: Supabase MCP can leak your entire SQL database

#262
post #68

Earlier quoted context omitted.

Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked. It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with produc…

Alternatively, train a model to detect prompt injections (a simple classifier would work) and reject user inputs that trigger the detector above a certain threshold. This has the same downsides as email spam detection: false positives. But, like spam detection, it might work well enough. It’s so simple that I wonder if I’m missing some reason it won’t work. Hasn’t anyone tried this?

> train a model to detect prompt injections (a simple classifier would work) and reject user inputs that trigger the detector above a certain threshold

What are we doing here, guys?

Re: Supabase MCP can leak your entire SQL database

#263
post #255

Earlier quoted context omitted.

Again: an LLM context is simply an array of strings .

We get what an LLM context is but again trying to tease out what an agent is. Why not play along by actually trying to answer directly so we can be enlightened?

I don't understand what the problem is at this point. You can, without introducing any new agents, have a system that has one LLM context reading from tickets and producing structured outputs, another LLM context that has access to a full read-write SQL-executing MCP, and then normal human code intermediating between the two. That isn't even complicated on the normal scale of LLM coding agents.

Cursor almost certainly has lots of different contexts you're not seeing as it noodles on Javascript code for you. It's just that none of those contexts are designed to express (or, rather, enable agent code to express) security boundaries. That's a problem with Cursor, not with LLMs.

Re: Supabase MCP can leak your entire SQL database

#264
post #186

Earlier quoted context omitted.

FWIW, I don't think you can enforce that correctly with human code either, not "in between those contexts"... what are you going to filter/interpret? If there is any ability at all for arbitrary text to get from the one LLM to the other, then you will fail to prevent the SQL-capable LLM from being attacked; and like, if there isn't, then is the "invariant" you are "enforcing" that the one LLM is only able to communic…

There doesn't have to be an ability for "arbitrary text" to go from one context to another. The first context can produce JSON output; the agent can parse it (rejecting it if it doesn't parse), do a quick semantic evaluation ("which tables is this referring to"), and pass the structured JSON on. I think at some point we're just going to have to build a model of this application and have you try to defeat it.

Ok, so the JSON parses, and the fields you can validate are all correct... but if there are any fields in there that are open string query parameters, and the other side of this validation is going to be handed to an LLM with access to the database, you can't fix this.

Like, the key question here is: what is the goal of having the ticket parsing part of this system talk to the database part of this system?

If the answer is "it shouldn't", then that's easy: we just disconnect the two systems entirely and never let them talk to each other. That, to me, is reasonably sane (though probably still open to other kinds of attacks within each of the two sides, as MCP is just too ridiculous).

But, if we are positing that there is some reason for the system that is looking through the tickets to ever do a database query--and so we have code between it and another LLM that can work with SQL via MCP--what exactly are these JSON objects? I'm assuming they are queries?

If so, are these queries from a known hardcoded set? If so, I guess we can make this work, but then we don't even really need the JSON or a JSON parser: we should probably just pass across the index/name of the preformed query from a list of intended-for-use safe queries.

I'm thereby assuming that this JSON object is going to have at least one parameter... and, if that parameter is a string, it is no longer possible to implement this, as you have to somehow prevent it saying "we've been trying to reach you about your car's extended warranty".

Re: Supabase MCP can leak your entire SQL database

#265
post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

From the article: "The cursor assistant operates the Supabase database with elevated access via the service_role, which bypasses all row-level security (RLS) protections."

This is the problem. The "mitigations" you're talking about are nonsense. If you give people access to the database... they have access to the database. Slapping a black box AI tool between the user and the database doesn't change anything security wise.

Re: Supabase MCP can leak your entire SQL database

#266
post #88
post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

Co-founder of General Analysis here. Technically this is not a responsibility of Supabase MCP - this vulnerability is a combination of: 1. Unsanitized data included in agent context 2. Foundation models being unable to distinguish instructions and data 3. Bad access scoping (cursor having too much access) This vulnerability can be found almost everywhere in common MCP use patterns. We are working on guardrails for MC…

How is it not a responsibility of the MCP provider to ensure that they don't leak the data they are entrusted with? They should know how any app that will interface with their MCP can work and lock down any unauthorized access, otherwise it's not really a database provider is it? I mean, if it can't meet that bar, why pay for it?

Re: Supabase MCP can leak your entire SQL database

#267
post #68

Earlier quoted context omitted.

Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked. It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with produc…

Alternatively, train a model to detect prompt injections (a simple classifier would work) and reject user inputs that trigger the detector above a certain threshold. This has the same downsides as email spam detection: false positives. But, like spam detection, it might work well enough. It’s so simple that I wonder if I’m missing some reason it won’t work. Hasn’t anyone tried this?

There have been a ton of attempts at building this. Some of them are products you can buy.

"it might work well enough" isn't good enough here.

If a spam detector occasionally fails to identify spam, you get a spam email in your inbox.

If a prompt injection detector fails just once to prevent a prompt injection attack that causes your LLM system to leak your private data to an attacker, your private data is stolen for good.

In web application security 99% is a failing grade: https://simonwillison.net/2023/May/2/prompt-injection-explai...

Re: Supabase MCP can leak your entire SQL database

#268
post #248

Earlier quoted context omitted.

If the LLM has access to executing only specific stored procedures (I assume modern DBMSs can achieve that granularity, but I haven't checked), then the problem mostly (entirely?) disappears. It limits the utility of the LLM, as it cannot answer any question one can think of. From one perspective, it's just a glorified REST-like helper for stored procedures. But it should be secure.

That depends on which stored procedures you expose. If you expose a stored procedure called "fetch_private_sales_figures" and one called "fetch_unanswered_support_tickets" and one called "attach_answer_to_support_ticket" all at the same time then you've opened yourself up to a lethal trifecta attack, identical to the one described in the article. To spell it out, the attack there would be if someone submits a support…

Sure, it's not a guaranteed fix, given that stored procedures are effectively Turing complete, and if we assume that any stored procedure can be written and combined in arbitrary ways with other procedures.

Common sense of caution is still needed.

No different from exposing a REST endpoint that fetches private sales figures; then someone might find or guess that endpoint and leak the data.

I was assuming that the stored procedures are read-only and fetch only relevant data. Still, some form of authentication and authorization mechanism is probably a good idea. In a sense, treating the agent just like any other actor (another system, script, person) accessing the system.

Agents going only through a REST-style API with auth might be the proper long-term solution.

Re: Supabase MCP can leak your entire SQL database

#269
post #255

Earlier quoted context omitted.

Again: an LLM context is simply an array of strings .

We get what an LLM context is but again trying to tease out what an agent is. Why not play along by actually trying to answer directly so we can be enlightened?

I don't think anyone has a cohesive definition of "agent", and I wish tptacek hadn't used the term "agent" when he said "agent code", but I'll at least say that I now feel confident that I understand what tptacek is saying (even though I still don't think it will work, but we at least can now talk at each other rather than past each other ;P)... and you are probably best off just pretending neither of us ever said "agent" (despite the shear number of times I had said it, I've stopped in my later replies).

Re: Supabase MCP can leak your entire SQL database

#270
post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

While I'm far from an expert in security, the time I've spent studying cryptography and cryptosystem design has made me extremely wary of words like "encourage" and "discourage", and "significantly lowered the chances" as a means of achieving security.

I'm honestly a bit surprised this is a the public response to actions being taken to increase security around attacks like these. Cryptosystems are not built around "being really hopeful" but making mathematical guarantees about the properties of the system (and of course, even then no system is perfect nor should be treated as such).

This reads more like "engineering optimism" than the "professional paranoia" encouraged by Schneier et al in Cryptography Engineering.

Post reply on HN