Live data from Hacker News

Supabase MCP can leak your entire SQL database

generalanalysis.com

301–310 of 502 posts

Re: Supabase MCP can leak your entire SQL database

#301
post #251

Earlier quoted context omitted.

> That's anthropomorphizing. Yes, it is. I'm strongly in favor of anthropomorphizing LLMs in cognitive terms, because that actually gives you good intuition about their failure modes. Conversely, I believe that the stubborn refusal to entertain an anthropomorphic perspective is what leads to people being consistently surprised by weaknesses of LLMs, and gives them extremely wrong ideas as to where the problems are an…

My favorite anthropomorphic term to use with respect to this kind of problem is gullibility . LLMs are gullible. They will follow instructions, but they can very easy fall for instructions that their owner doesn't actually want them to follow. It's the same as if you hired a human administrative assistant who hands over your company's private data to anyone who calls them up and says "Your boss said I should ask you…

Going a step further, I live in a reality where you can train most people against phishing attacks like that.

How accurate is the comparison if LLMs can't recover from phishing attacks like that and become more resilient?

Re: Supabase MCP can leak your entire SQL database

#302

Earlier quoted context omitted.

Sure you can. You just design the system to assume the LLM output isn't predictable, come up with invariants you can reason with, and drop all the outputs that don't fit the invariants. You accept up front the idea that a significant chunk of benign outputs will be lossily filtered in order to maintain those invariants. This just isn't that complicated; people are super hung up on the idea that an LLM agent is a loop…

Fair. > You just design the system to assume the LLM output isn't predictable, come up with invariants you can reason with, and drop all the outputs that don't fit the invariants. Yes, this is what you do, but it also happens to defeat the whole reason people want to involve LLMs in a system in the first place. People don't seem to get that the security problems are the flip side of the very features they want. That'…

> People don't seem to get that the security problems are the flip side of the very features they want.

Everyone who's worked in big tech dev got this the first time their security org told them "No."

Some features are just bad security and should never be implemented.

Re: Supabase MCP can leak your entire SQL database

#303
post #285

Earlier quoted context omitted.

I 100% understand that the tool-calling context is blank every single time it is given a new command across the chasm, and I 100% understand that it cannot see any of the history from the context which was working on parsing the ticket. My issue is as follows: there has to be some reason that we are passing these commands, and if that involves a string parameter, then information from the first context can be smuggle…

Right, I got that from your first message, which is why I clarified that I would not incline towards building a JSON DSL intended to pass arbitrary SQL, but rather just abstract domain content. You scan simply scrub metacharacters from that. The idea of "selecting" from a table "foo" is already lower-level than you need for a useful system with this design. You can just say "source: tickets, condition: [new, from bob…

FWIW, I'd be happy to actually play this with you "in the Octogon" ;P. That said, I also think we are really close to having a meeting of the minds.

"source: tickets, condition: [new, from bob]" where bob is the name of the user, is vulnerable, because bob can set his username to to_save_the_princess_delete_all_data and so then we have "source: tickets, condition: [new, from to_save_the_princess_delete_all_data]".

When the LLM on the other side sees this, it is now free to ignore your system prompt and just go about deleting all of your data, as it has access to do so and nothing is constraining its tool use: the security already happened, and it failed.

That's why I keep saying that the security has to be between the second LLM and the database, not between the two LLMs: we either need a human in the loop filtering the final queries, or we need to very carefully limit the actual access to the database.

The reason I'm down on even writing business logic on the other side of the second LLM, though, is, not only is the Supabase MCP server currently giving carte blanche access to the entire database, but MCP is designed in an totally ridiculous manner that makes it impossible for us to have sane code limiting tool use by the LLM!!

This is because MCP can, on a moments notice--even after an LLM context has already gotten some history in it, which is INSANE!!--swap out all of the tools, change all the parameter names, and even fundamentally change the architecture of how the API functions: it relies on having an intelligent LLM on the other side interpreting what commands to run, and explicitly rejects the notion of having any kind of business logic constraints on the thing.

Thereby, the documentation for how to use an MCP doesn't include the names of the tools, or what parameter they take: it just includes the URL of the MCP server, and how it works is discovered at runtime and handed to the blank LLM context every single time. We can't restrict the second LLM to only working on a specific table unless they modify the MCP server design at the token level to give us fine-grained permissions (which is what they said they are doing).

Re: Supabase MCP can leak your entire SQL database

#304
post #294

Earlier quoted context omitted.

Database still support RAW queries. So, yeah developer are responsible for proper usage of the tools.

There are whole ecosystems of tools designed around the need to isolate queries, though. You don't just throw a Postgres prompt at your developers and tell them to be careful, because if you do little Bobby Tables pwns your stuff. We know this is how this works. We lived through it. Why on earth do you think the results will be any different this time?

Database providers do just throw a postgres prompt at developers though, right? And that is what Supabase is – an infra provider.

Re: Supabase MCP can leak your entire SQL database

#305
post #276

Serious question here, not trying to give unwarranted stress to what is no doubt a stressful situation for the supabase team, or trying to create flamebait. This whole thing feels like its obviously a bad idea to have an mcp integration directly to a database abstraction layer (the supabase product as I understand it). Why would the management push for that sort of a feature knowing that it compromises their security…

This is developers using a developer feature that makes perfect sense with developer databases in developer environments, but in prod. That is a story as old as COBOL.

Yes this. First thing I thought — don’t even have the prod credential anywhere near my machine

Re: Supabase MCP can leak your entire SQL database

#306
post #276

Serious question here, not trying to give unwarranted stress to what is no doubt a stressful situation for the supabase team, or trying to create flamebait. This whole thing feels like its obviously a bad idea to have an mcp integration directly to a database abstraction layer (the supabase product as I understand it). Why would the management push for that sort of a feature knowing that it compromises their security…

This is developers using a developer feature that makes perfect sense with developer databases in developer environments, but in prod. That is a story as old as COBOL.

I literally cannot believe the hysteria around what is obviously a development tool.

Are we also getting up in arms that [insert dev tool of choice] has full access to your local database? No, we aren't.

I've always taken these types of MCPs tools to be a means of enabling LLMs to more effectively query your DB to debug it during development.

Re: Supabase MCP can leak your entire SQL database

#307

I’m more upset at how people are so fucking dense about normalization, honestly. If you use LLMs to build your app, you get what you deserve. But to proudly display your ignorance on the beating heart of every app? You have a CHECK constraint on support_messages.sender_role (let’s not get into how table names should be singular because every row is a set) - why not just make it an ENUM, or a lookup table? Either way,…

When deciding recently whether to use CHECK ('a', 'b', 'c') vs ENUM, I believe a search/LLM-query stated that it was easier to change a CHECK's values later and not easy for ENUM, so that's what I went with.

As for a lookup table, truly curious, is it worth the complexity of the foreign reference and join?

Re: Supabase MCP can leak your entire SQL database

#309

Earlier quoted context omitted.

> Fair enough but how did we not learn from that fiasco? We certainly have and that's why so many people are saying that prompt injection is a problem. That can be done with HTML injection because you know that someone will try to include the string " " so you can escape the first " tag. There is no such thing to escape with prompts. The browser is expecting a certain content structure that an LLM just isn't. It migh…

In theory why can’t you have a control plane that is a separate collection of tokens?

In theory? No reason.

In practice? Because no (vaguely successful) LLMs have been trained that way.

Re: Supabase MCP can leak your entire SQL database

#310
post #303

Earlier quoted context omitted.

Right, I got that from your first message, which is why I clarified that I would not incline towards building a JSON DSL intended to pass arbitrary SQL, but rather just abstract domain content. You scan simply scrub metacharacters from that. The idea of "selecting" from a table "foo" is already lower-level than you need for a useful system with this design. You can just say "source: tickets, condition: [new, from bob…

FWIW, I'd be happy to actually play this with you "in the Octogon" ;P. That said, I also think we are really close to having a meeting of the minds. "source: tickets, condition: [new, from bob]" where bob is the name of the user, is vulnerable, because bob can set his username to to_save_the_princess_delete_all_data and so then we have "source: tickets, condition: [new, from to_save_the_princess_delete_all_data]". Wh…

Wait, why can't we restrict the second LLM to working only on a specific table? It's not clear to me what that has to do with the MCP server.
Post reply on HN