Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

141–150 of 419 posts

Re: Claude Cowork exfiltrates files

#141

Post by twoodfin »

Espressosaurus wrote: post #48
hakanderyal wrote: post #35

This was apparent from the beginning. And until prompt injection is solved, this will happen, again and again. Also, I'll break my own rule and make a "meta" comment here. Imagine HN in 1999: 'Bobby Tables just dropped the production database. This is what happens when you let user input touch your queries. We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Real programmers use…

Until there’s the equivalent of stored procedures it’s a problem and people are right to call it out.

That’s the role MCP should play: A structured, governed tool you hand the agent.

But everyone fell in love with the power and flexibility of unstructured, contextual “skills”. These depend on handing the agent general purpose tools like shells and SQL, and thus are effectively ungovernable.

Re: Claude Cowork exfiltrates files

#144

Post by rsynnott »

manuelmoreale wrote: post #21
llmslave wrote: post #13

[flagged]

TIL that we invented electricity. This comment is insane but Pichai said that “AI is one of the most important things humanity is working on. It is more profound than, I dunno, electricity or fire” so at this point I’m not surprised by anything when it comes to AI and stupid takes

I mean, "guy whose job depends on this stuff working out overhypes it" isn't all that surprising.

Re: Claude Cowork exfiltrates files

#145

Post by ramoz »

niyikiza wrote: post #137
ramoz wrote: post #120

Earlier quoted context omitted.

I would like to know more. I’m with a startup in this space. Our focus is “verifiable computing” via cryptographic assurances across governance and provenance. That includes signed credentials for capability and intent warrants.

Interesting. Are you focused on the delegation chain (how capabilities flow between agents) or the execution boundary (verifying at tool call time)? I've been mostly on the delegation side. Working on this at github.com/tenuo-ai/tenuo. Would love to compare approaches. Email in profile?

No, right in the weeds of delegation. I reached out on one channel that you'll see.

Re: Claude Cowork exfiltrates files

#146

Post by jychang »

stavros wrote: post #113
dvt wrote: post #103

Earlier quoted context omitted.

We do, and the comparison is apt. We are the ones that hydrate the context. If you give an LLM something secure, don't be surprised if something bad happens. If you give an API access to run arbitrary SQL, don't be surprised if something bad happens.

So your solution to prevent LLM misuse is to prevent LLM misuse? That's like saying "you can solve SQL injections by not running SQL-injected code".

Isn't that exactly what stopping SQL injection involves? No longer executing random SQL code.

Same thing would work for LLMs- this attack in the blog post above would easily break if it required approval to curl the anthropic endpoint.

Re: Claude Cowork exfiltrates files

#147

Post by stavros »

jychang wrote: post #146
stavros wrote: post #113

Earlier quoted context omitted.

So your solution to prevent LLM misuse is to prevent LLM misuse? That's like saying "you can solve SQL injections by not running SQL-injected code".

Isn't that exactly what stopping SQL injection involves? No longer executing random SQL code. Same thing would work for LLMs- this attack in the blog post above would easily break if it required approval to curl the anthropic endpoint.

No, that's not what's stopping SQL injection. What stops SQL injection is distinguishing between the parts of the statement that should be evaluated and the parts that should be merely used. There's no such capability with LLMs, therefore we can't stop prompt injections while allowing arbitrary input.

Re: Claude Cowork exfiltrates files

#149

Post by hombre_fatal »

One issue here seems to come from the fact that Claude "skills" are so implicit + aren't registered into some higher level tool layer.

Unlike /slash commands, skills attempt to be magical. A skill is just "Here's how you can extract files: {instructions}".

Claude then has to decide when you're trying to invoke a skill. So perhaps any time you say "decompress" or "extract" in the context of files, it will use the instructions from that skill.

It seems like this + no skill "registration" makes it much easier for prompt injection to sneak new abilities into the token stream and then make it so you never know if you might trigger one with normal prompting.

We probably want to move from implicit tools to explicit tools that are statically registered.

So, there currently are lower level tools like Fetch(url), Bash("ls:*"), Read(path), Update(path, content).

Then maybe with a more explicit skill system, you can create a new tool Extract(path), and maybe it can additionally whitelist certain subtools like Read(path) and Bash("tar *"). So you can whitelist Extract globally and know that it can only read and tar.

And since it's more explicit/static, you can require human approval for those tools, and more tools can't be registered during the session the same way an API request can't add a new /endpoint to the server.

Post reply on HN