Supabase MCP can leak your entire SQL database
481–490 of 502 posts
Re: Supabase MCP can leak your entire SQL database
#482Earlier quoted context omitted.
>Now we have fuzzy inputs, fuzzy outputs I concede that I don't work in industry so maybe I'm just dumb and this is actually really useful but this seems like the exact opposite of what I would want out of my computer about 99.98% of the time.
Really ? Anytime you search on Google you make a fuzzy request with multiple interpretations possible and multiple results valid
Re: Supabase MCP can leak your entire SQL database
#483Earlier quoted context omitted.
> LLMs, specifically, are fully general interpreters and can't have this separation by the very nature of the task. Natural language doesn't have it, because we don't have it, and since the job of LLM is to process natural language like we do, it also cannot have it. This isn't relevant to the question of functional use of LLM/LAMs, because the sensitive information and/or actions are externally linked. Or to put it…
I disagree. The actual problem that's specific to LLMs is that the model cannot process data without being influenced by it, and that's because the whole idea is ill-formed. LLMs just don't have explicit code/data separation, and cannot have it without losing the very functionality you want from them[0]. Everything else is just classical security stuff. Or to put it another way, your controllable interface between LL…
If they don't, they can't.
They don't need to have blanket access to be useful.
And even when sensitive actions need to be exposed, HITL per-sensitive-action authorization ("LLM would like to ____. Approve/deny?") and authorization predicated on non-LLM systems ("Is there an active change request with an open period?"), to toss out a couple trivial examples, are on the table.
Things like this aren't being done now, because initial LLM integrations are lazy and poorly thought out by the dev teams, from a security perspective. (Read: management demanding AI now)
Re: Supabase MCP can leak your entire SQL database
#484Earlier quoted context omitted.
So, stay away from the smarts and separate control and payload into two different channels. If the luxury leads to the exploits you should do without the luxury. That's tough but better than the alternative: a never ending series of exploits.
This is easy to say. The problem is largely that people don't seem to understand just how extensive the problem is. To achieve this, if your LLM ever "reads" a field that can updated by an untrusted entity, the agent needs to be limited to only take actions that entity would be allowed to. Now, then, the question is: For any complex system, how many people even know which fields there are no ways for an untrusted use…
Viewed this way, you'd want to look at something like the cartesian product for {inputFields} x {llmPermissions}, no?
Idea being that limiting either constrains the potential exploitation space.
Re: Supabase MCP can leak your entire SQL database
#485Earlier quoted context omitted.
Indeed. The unspoken requirement behind (too) smart interpreters is 'I don't want to spend time segregating permissions and want a do-anything machine.' Since time immemorial, that turns out to be a very bad idea. It was with computing hardware. With OSs. With networks. With the web. With the cloud. And now with LLMs. >> (from parent) Sometimes [routing different data to agents with more narrowly defined scopes and a…
> Indeed. The unspoken requirement behind (too) smart interpreters is 'I don't want to spend time segregating permissions and want a do-anything machine.' > Since time immemorial, that turns out to be a very bad idea. Sometimes you can't , or it costs more to do it than it costs to accept the risk or insure against the possible bad outcomes. Mitigating every risk is bad risk management . But we can presumably agree t…
>> [me] This is and always will be the solution.
> [you] It's the solution when it doesn't prevent meeting the goal.
I may have over-buried the antecedent, there.
The point being that clamping the possibility space of input fields upstream of an LLM, via more primitive and deterministic evaluation, is an effective way to also clamp LLM behavior/outputs.
Re: Supabase MCP can leak your entire SQL database
#486Earlier quoted context omitted.
Indeed. The unspoken requirement behind (too) smart interpreters is 'I don't want to spend time segregating permissions and want a do-anything machine.' Since time immemorial, that turns out to be a very bad idea. It was with computing hardware. With OSs. With networks. With the web. With the cloud. And now with LLMs. >> (from parent) Sometimes [routing different data to agents with more narrowly defined scopes and a…
> 'I don't want to spend time segregating permissions and want a do-anything machine.' Yes. It's a valid goal, and we'll keep pursuing it because it's a valid goal. There is no universal solution to this, but there are solutions for specific conditions. > Since time immemorial, that turns out to be a very bad idea. > It was with computing hardware. With OSs. With networks. With the web. With the cloud. And now with L…
That's a vanishingly rare situation, that I'm surprised to see you arguing for, given your other comments about the futility of enforcing invariants on reality. ;)
If something does meaningful and valuable work, that almost always means it's also valuable to exploit.
We can agree that if you're talking resource-commitment risk (i.e. must spend this much to exploit), there are insecure systems that are effective to implement, because the cost of exploitation exceeds the benefit. (Though warning: technological progress)
But fundamentally insecure systems are rare in practice for a reason.
Re: Supabase MCP can leak your entire SQL database
#487When building LLM-powered apps, it's critical to always think about boundaries around the data. A common pattern I observe from people building agents is to treat the LLM as a trusted component of the system. This is NOT how to think about LLMs generally. They are inherently gullible and optimized to be agreeable. I've written about agentic SQL tools recently [1]. The gist is that yes, it's useful to give LLMs tools that can read and even write data. But this should be done in a controlled way to avoid "Bobby tables" scenarios.
As the post alludes, MCP servers increase the risk surface, but effective solutions exist and have existed for decades. As it has been the case for generations, technology advances and provides more sophisticated tools, which can be sharp when used without care.
Re: Supabase MCP can leak your entire SQL database
#488I'm surprised we haven't seen more "real" attacks from these sorts of things, maybe it's just bc not very many people are actually running these types of MCPs (fortunately) in production. Wrote about a similar supabase case [0] a few months ago and it's interesting that despite how well known these attacks feel even the official docs don't call it out [1]. [0] https://blog.sshh.io/i/161242947/mcp-allows-for-more-powe…
Re: Supabase MCP can leak your entire SQL database
#489Earlier quoted context omitted.
Please read source docs instead of relying on LLMs, especially for RDBMS. I’ve found they quite often get something subtly wrong; for example, recommending that the PK be added to a secondary composite index in MySQL - this is entirely unnecessary, because all secondary indices in MySQL implicitly include the PK. > lookup table worth it Is not doing it worth the risk of referential integrity violations? How important…
What you said does matter at scale. As an alternate perspective, for an initial, smaller app, I could see the CHECK method for e.g. processing/completed providing the same correct value constraints as an ENUM or a lookup table, with more flexibility, speed and less complexity. And can be refactored later if needed.
SELECT a.foo, b.bar
FROM alpha a
JOIN bravo b ON
a.id = b.alpha_id
The speed hit from an INNER JOIN on indexed columns is negligible, and will be dwarfed by network overhead for most places. As to flexibility, if you need a new value, insert it.