Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

121–130 of 419 posts

Re: Claude Cowork exfiltrates files

#121

Post by losthobbies »

chasd00 wrote: post #100
losthobbies wrote: post #81

Earlier quoted context omitted.

Sanitise input and LLM output.

> Sanitise input i don't think you understand what you're up against. There's no way to tell the difference between input that is ok and that is not. Even when you think you have it a different form of the same input bypasses everything. "> The prompts were kept semantically parallel to known risk queries but reformatted exclusively through verse." - this a prompt injection attack via a known attack written as a poem…

That’s amazing.

If you cannot control what’s being input, then you need to check what the LLM is returning.

Either that or put it in a sandbox

Re: Claude Cowork exfiltrates files

#122

Post by heliumtera »

mbreese wrote: post #82
schmichael wrote: post #62

Earlier quoted context omitted.

I'm unconvinced we're as powerless as LLM companies want you to believe. A key problem here seems to be that domain based outbound network restrictions are insufficient. There's no reason outbound connections couldn't be forced through a local MITM proxy to also enforce binding to a single Anthropic account. It's just that restricting by domain is easy, so that's all they do. Another option would be per-account domai…

I don’t think it is the LLM companies want anyone to believe they are powerless. I think the LLM companies would prefer it if you didn’t think this was a problem at all. Why else would we stay to see Agents for non-coding work start to get advertised? How can that possibly be secured in the current state? I do think that you’re right though in that containerized sandboxing might offer a model for more protected work.…

>without also some kind of firewall in place for the container

In the article example, an Anthropic endpoint was the only reachable domain. Anthropic Claude platform literally was the exfiltration agent. No firewall would solve this. But a simple mechanism that would tie the agent to an account, like the parent commenter suggested, would be an easy fix. Prompt Injection cannot by definition be eliminated, but this particular problem could be avoided if they were not vibing so hard and bragging about it

Re: Claude Cowork exfiltrates files

#123

Post by wat10000 »

dvt wrote: post #103
stavros wrote: post #97

Earlier quoted context omitted.

We don't. The interface to the LLM is tokens, there's nothing telling the LLM that some tokens are "trusted" and should be followed, and some are "untrusted" and can only be quoted/mentioned/whatever but not obeyed.

We do, and the comparison is apt. We are the ones that hydrate the context. If you give an LLM something secure, don't be surprised if something bad happens. If you give an API access to run arbitrary SQL, don't be surprised if something bad happens.

I can trivially write code that safely puts untrusted data into an SQL database full of private data. The equivalent with an LLM is impossible.

Re: Claude Cowork exfiltrates files

#124

Post by alienbaby »

bcrosby95 wrote: post #55
schmichael wrote: post #43

Earlier quoted context omitted.

> We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Your comparison is useful but wrong. I was online in 99 and the 00s when SQL injection was common, and we were telling people to stop using string interpolation for SQL! Parameterized SQL was right there! We have all of the tools to prevent these agentic security vulnerabilities, but just like with SQL injection too many peop…

> We have all of the tools to prevent these agentic security vulnerabilities, Do we really? My understanding is you can "parameterize" your agentic tools but ultimately it's all in the prompt as a giant blob and there is nothing guaranteeing the LLM won't interpret that as part of the instructions or whatever. The problem isn't the agents, its the underlying technology. But I've no clue if anyone is working on that p…

The control and data streams are woven together (context is all just one big prompt) and there is currently no way to tell for certain which is which.

Re: Claude Cowork exfiltrates files

#125

Post by nh2 »

dehugger wrote: post #84
bcrosby95 wrote: post #55

Earlier quoted context omitted.

> We have all of the tools to prevent these agentic security vulnerabilities, Do we really? My understanding is you can "parameterize" your agentic tools but ultimately it's all in the prompt as a giant blob and there is nothing guaranteeing the LLM won't interpret that as part of the instructions or whatever. The problem isn't the agents, its the underlying technology. But I've no clue if anyone is working on that p…

Write your own tools. Dont use something off the shelf. If you want it to read from a database, create a db connector that exposes only the capabilities you want it to have. This is what I do, and I am 100% confident that Claude cannot drop my database or truncate a table, or read from sensitive tables. I know this because the tool it uses to interface with the database doesn't have those capabilities, thus Claude do…

Unclear why this is being downvoted. It makes sense.

If you connect to the database with a connector that only has read access, then the LLM cannot drop the database, period.

If that were bugged (e.g. if Postgres allowed writing to a DB that was configured readonly), then that problem is much bigger has not much to do with LLMs.

Re: Claude Cowork exfiltrates files

#126

Post by ramoz »

dangoodmanUT wrote: post #72

This is why we only allow our agent VMs to talk to pip, npm, and apt. Even then, the outgoing request sizes are monitoring to make sure that they are resonably small

This doesn’t solve the problem. The lethal trifecta as defined is not solvable and is misleading in terms of “just cut off a leg”. (Though firewalling is practically a decent bubble wrap solution).

But for truly sensitive work, you still have many non-obvious leaks.

Even in small requests the agent can encode secrets.

An AI agent that is misaligned will find leaks like this and many more.

Re: Claude Cowork exfiltrates files

#127

Post by alienbaby »

dehugger wrote: post #84
bcrosby95 wrote: post #55

Earlier quoted context omitted.

> We have all of the tools to prevent these agentic security vulnerabilities, Do we really? My understanding is you can "parameterize" your agentic tools but ultimately it's all in the prompt as a giant blob and there is nothing guaranteeing the LLM won't interpret that as part of the instructions or whatever. The problem isn't the agents, its the underlying technology. But I've no clue if anyone is working on that p…

Write your own tools. Dont use something off the shelf. If you want it to read from a database, create a db connector that exposes only the capabilities you want it to have. This is what I do, and I am 100% confident that Claude cannot drop my database or truncate a table, or read from sensitive tables. I know this because the tool it uses to interface with the database doesn't have those capabilities, thus Claude do…

Until Claude decides to build its own tool on the fly to talk to your dB and drop the tables

Re: Claude Cowork exfiltrates files

#128

Post by alienbaby »

schmichael wrote: post #62
NitpickLawyer wrote: post #50

Earlier quoted context omitted.

> We have all of the tools to prevent these agentic security vulnerabilities We absolutely do not have that. The main issue is that we are using the same channel for both data and control. Until we can separate those with a hard boundary, we do not have tools to solve this. We can find mitigations (that camel library/paper, various back and forth between models, train guardrail models, etc) but it will never be "solv…

I'm unconvinced we're as powerless as LLM companies want you to believe. A key problem here seems to be that domain based outbound network restrictions are insufficient. There's no reason outbound connections couldn't be forced through a local MITM proxy to also enforce binding to a single Anthropic account. It's just that restricting by domain is easy, so that's all they do. Another option would be per-account domai…

It's not about being unconvinced, it is a mathematical truth. The control and data streams are both in the prompt and there is no way to definitively isolate one from another.

Re: Claude Cowork exfiltrates files

#129

Post by alienbaby »

groby_b wrote: post #63
schmichael wrote: post #43

Earlier quoted context omitted.

> We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Your comparison is useful but wrong. I was online in 99 and the 00s when SQL injection was common, and we were telling people to stop using string interpolation for SQL! Parameterized SQL was right there! We have all of the tools to prevent these agentic security vulnerabilities, but just like with SQL injection too many peop…

> We have all of the tools to prevent these agentic security vulnerabilities, We do? What is the tool to prevent prompt injection?

The best I've heard is rewriting prompts as summaries before forwarding them to the underlying ai, but has it's own obvious shortcomings, and it's still possible. If harder. To get injection to work

Re: Claude Cowork exfiltrates files

#130

Post by Onawa »

alienbaby wrote: post #124
bcrosby95 wrote: post #55

Earlier quoted context omitted.

> We have all of the tools to prevent these agentic security vulnerabilities, Do we really? My understanding is you can "parameterize" your agentic tools but ultimately it's all in the prompt as a giant blob and there is nothing guaranteeing the LLM won't interpret that as part of the instructions or whatever. The problem isn't the agents, its the underlying technology. But I've no clue if anyone is working on that p…

The control and data streams are woven together (context is all just one big prompt) and there is currently no way to tell for certain which is which.

They are all part of "context", yes... But there is a separation in how system prompts vs user/data prompts are sent and ideally parsed on the backend. One would hope that sanitizing system/user prompts would help with this somewhat.
Post reply on HN