Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supabase MCP can leak your entire SQL database

generalanalysis.com

Post reply on HN
Print view View original

391–400 of 502 posts

Re: Supabase MCP can leak your entire SQL database

#391

Post by egozverev »

Academic researcher here working on this exact issue. Prompt engineering methods are no sufficient to address the challenge. People in Academy and Industry labs are aware of the issue and actively working on it, see for instance:

[1] Camel: work by google deepmind on how to (provably!) prevent agent planner from being prompt-injected: https://github.com/google-research/camel-prompt-injection

[2] FIDES: similar idea by Microsoft, formal guarantees: https://github.com/microsoft/fides

[3] ASIDE: marking non-executable parts of input and rotating their embedding by 90 degrees to defend against prompt injections: https://github.com/egozverev/aside

[4] CachePrune: pruning attention matrices to remove "instruction activations" on prompt injections: https://arxiv.org/abs/2504.21228

[5] Embedding permission tokens and inserting them to prompts: https://arxiv.org/abs/2503.23250

Here's (our own) paper discussing why prompt based methods are not going to work to solve the issue: "Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?" https://arxiv.org/abs/2403.06833

Do not rely on prompt engineering defenses!

Re: Supabase MCP can leak your entire SQL database

#392

Post by [deleted] »

gregnr wrote: post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

[deleted]

Re: Supabase MCP can leak your entire SQL database

#393

Post by TeMPOraL »

Dylan16807 wrote: post #286
TeMPOraL wrote: post #247

Earlier quoted context omitted.

> If you tell your computer to print out a string, the data does control what the computer does, but in an extremely bounded way where you can make assertions about what happens! You'd like that to be true, but the underlying code has to actually constrain the system behavior this way, and it gets more tricky the more you want the system to do. Ultimately, this separation is a fake reality that's only as strong as th…

The ability to deliberately decide to ignore the boundary between code and data doesn't mean the separation rule isn't still separating. In the lab example, the person is worried and trying to do the right thing, but they know it's not part of the transcription task.

The point is, there is no hard boundary. The LLM too may know[0] following instructions in data isn't part of transcription task, and still decide to do it.

--

[0] - In fact I bet it does, in the sense that, doing something like Anthropic did[1], you could observe relevant concepts being activated within the model. This is similar to how it turned out the model is usually aware when it doesn't know the answer to a question.

[1] - https://www.anthropic.com/news/tracing-thoughts-language-mod...

Re: Supabase MCP can leak your entire SQL database

#394

Post by pmontra »

gregnr wrote: post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

You write about mitigations and I'm afraid that you are correct. Can any method be more than just a mitigation? When we give read access to something to somebody we can expect that only loyalty (or fear, or... but let's stick with loyalty) prevents that person from leaking information to other parties.

Improvements to prompting might increase the LLM equivalent of loyalty but people will always be creative at finding ways to circumvent limitations.

The only way not to lower security seems to be giving access to those LLMs only to the people that already have read access to the whole database. If it leaks all the the data to them, they could more easily have dumped it with traditional tools. This might make an LLM almost useless but if the LLM might be equivalent to a tool with superuser access, that's it.

Re: Supabase MCP can leak your entire SQL database

#395

Post by TeMPOraL »

magicalhippo wrote: post #246
TeMPOraL wrote: post #179

Earlier quoted context omitted.

That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing. What we call code, and what we call data, is just a question of convenience. For example, when editing or copying WMF files, it's convenient to think of them as data (mix of raster and vector graphics) - however, at least in the original implementation,…

> There is no separation of code and data on the wire - everything is a stream of bytes. There isn't one in electronics either - everything is signals going down the wires. Overall I agree with your message, but I think you're stretching it too far here. You can make code and data physically separate[1]. But if you then upload an interpreter, that "one level of abstraction up", you can mix code and data again. https:…

> Overall I agree with your message, but I think you're stretching it too far here. You can make code and data physically separate[1].

You cannot. I.e. this holds only within the abstraction level of the system. Not only it can be defeated one level up, as you illustrated, but also by going one or more levels down. That's where "side channels" come from.

But the most relevant part for this discussion is, even with something like Harvard architecture underneath, your typical software systems is defined in terms of reality several layers of abstraction above hardware - and LLMs, specifically, are fully general interpreters and can't have this separation by the very nature of the task. Natural language doesn't have it, because we don't have it, and since the job of LLM is to process natural language like we do, it also cannot have it.

Re: Supabase MCP can leak your entire SQL database

#396

Post by Orygin »

reddalo wrote: post #368
skinner927 wrote: post #327

Earlier quoted context omitted.

Microsoft’s cloud gets hacked multiple times a year, nobody cares. Everyone is connecting everything together. Business people with no security training/context are “writing” integrations with Lego-like services (and now LLMs). Cloudflare hiccups and the Internet crashes. Nobody cares about the things you’re saying anymore (I do!!). Extract more money. Move faster. Outcompete. Fix it later. Just get a bigger cyber in…

>Microsoft’s cloud gets hacked multiple times a year What cloud? Private SharePoint instances? Accounts? Free Outlook accounts? Do you have any source on this?

Small sample: https://www.virtru.com/blog/industry-updates/microsoft-data-...

I also can't find the news, but they were hacked a few years ago and the hackers were still inside their network for months while they were trying to get them out. I wouldn't trust anything from MS as most of their system is likely infected in some form

Re: Supabase MCP can leak your entire SQL database

#397

Post by TeMPOraL »

Traubenfuchs wrote: post #344
TeMPOraL wrote: post #179

Earlier quoted context omitted.

That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing. What we call code, and what we call data, is just a question of convenience. For example, when editing or copying WMF files, it's convenient to think of them as data (mix of raster and vector graphics) - however, at least in the original implementation,…

> There is no natural separation between code and data. They are the same thing. Seems there is a pretty clear distinction in the context of prepared statements.

It's an engineered distinction; it's only as good as the underlying code that enforces it, and only exists within the scope of that code.

Re: Supabase MCP can leak your entire SQL database

#398

Post by [deleted] »

gregnr wrote: post #43

Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections: - Encourage folks to use read-only by default in our docs [1] - Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2] - Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2] We noticed that t…

[deleted]

Re: Supabase MCP can leak your entire SQL database

#399

Post by ttoinou »

LambdaComplex wrote: post #216
tptacek wrote: post #68

Earlier quoted context omitted.

Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked. It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with produc…

Right? "Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data?" The entire point of programming is that (barring hardware failure and compiler bugs) the computer will always do exactly what it's told, and now progress apparently looks like having to "discourage" the computer from doing things and hoping that it listens?

   The entire point of programming is that (barring hardware failure and compiler bugs) the computer will always do exactly what it's told
New AI tech is not like regular programming we had before. Now we have fuzzy inputs, fuzzy outputs

Re: Supabase MCP can leak your entire SQL database

#400

Post by tudorg »

Another way to mitigate this is to make the agents always work only with a copy of the data that is anonymized. Assuming the anonymisation step removes / replaces all sensitive data, then whatever the AI agent does, it won't be disastrous.

The anonymization can be done by pgstream or pg_anonymizer. In combination with copy-on-write branching, you can create a safe environments on the fly for AI agents that get access to data relevant for production, but not quite production data.

Post reply on HN