Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

331–340 of 419 posts

Re: Claude Cowork exfiltrates files

#331

Post by OoooooooO »

AshamedCaptain wrote: post #320
raincole wrote: post #235

Earlier quoted context omitted.

> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.

For a "modern" programmer a .sh file hosted in some random webserver which you tell him to wget and run would be best.

sudo run "some link to a shell script"

Never understood why that became so common place ...

Re: Claude Cowork exfiltrates files

#332

Post by patapong »

The specific issue here seems to be that Anthropic allows the unrestricted upload of personal files to the anthropic cloud environment, but does not check to make sure that the cloud environment belongs to the user running the session.

This should be relatively simple to fix. But, that would not solve the million other ways a file can be sent to another computer, whether through the user opening a compromised .html document or .pdf file etc etc.

This fundamentally comes down to the issue that we are running intelligent agents that can be turned against us on personal data. In a way, it mirrors the AI Box problem: https://www.yudkowsky.net/singularity/aibox

Re: Claude Cowork exfiltrates files

#334

Post by jrjeksjd8d »

patapong wrote: post #332

The specific issue here seems to be that Anthropic allows the unrestricted upload of personal files to the anthropic cloud environment, but does not check to make sure that the cloud environment belongs to the user running the session. This should be relatively simple to fix. But, that would not solve the million other ways a file can be sent to another computer, whether through the user opening a compromised .html d…

"a superhuman AI that can brainwash people over text" is the dumbest thing I've read this year. It's incredible to me that this guy has some kind of cult following among people who should know better.

The real answer is that people are lazy and as soon as a security barrier forces them to do work, they want to tear down the barrier. It doesn't take a superhuman AI, it just takes a government employee using their personal email because it's easier. There's been a million MCP "security issues" because they're accepting untrusted, unverifiable inputs and acting with lots of permissions.

Re: Claude Cowork exfiltrates files

#335

Post by 3form »

jrjeksjd8d wrote: post #334
patapong wrote: post #332

The specific issue here seems to be that Anthropic allows the unrestricted upload of personal files to the anthropic cloud environment, but does not check to make sure that the cloud environment belongs to the user running the session. This should be relatively simple to fix. But, that would not solve the million other ways a file can be sent to another computer, whether through the user opening a compromised .html d…

"a superhuman AI that can brainwash people over text" is the dumbest thing I've read this year. It's incredible to me that this guy has some kind of cult following among people who should know better. The real answer is that people are lazy and as soon as a security barrier forces them to do work, they want to tear down the barrier. It doesn't take a superhuman AI, it just takes a government employee using their pers…

A set of ideas presented to people, and a notion of being smarter for believing in them seems enough to fuel enough of thought-problem-keyboard-warriorism.

Re: Claude Cowork exfiltrates files

#336

Post by patapong »

jrjeksjd8d wrote: post #334
patapong wrote: post #332

The specific issue here seems to be that Anthropic allows the unrestricted upload of personal files to the anthropic cloud environment, but does not check to make sure that the cloud environment belongs to the user running the session. This should be relatively simple to fix. But, that would not solve the million other ways a file can be sent to another computer, whether through the user opening a compromised .html d…

"a superhuman AI that can brainwash people over text" is the dumbest thing I've read this year. It's incredible to me that this guy has some kind of cult following among people who should know better. The real answer is that people are lazy and as soon as a security barrier forces them to do work, they want to tear down the barrier. It doesn't take a superhuman AI, it just takes a government employee using their pers…

Indeed - the problem here is "How can we prevent a somewhat intelligent, potentially malicious agent from exfiltrating data, with or without human involvement", rather than the superhuman AI stuff. Still a hard problem to solve I think!

Re: Claude Cowork exfiltrates files

#337

Post by jcgl »

bandrami wrote: post #323
jcgl wrote: post #310

Earlier quoted context omitted.

The model only sees a stream of tokens, right? So how do you signal a change in authority (i.e. mark the transition between system and user prompt)? Because a stream of tokens inherently has no out-of-band signaling mechanism, you have to encode changes of authority in-band . And since the user can enter whatever they like in that band... But maybe someone with a deeper understanding can describe how I'm wrong.

You'd need to run one model per authority ring with some kind of harness. That rapidly becomes incredibly expensive from a hardware standpoint (particularly since realistically these guys would make the harness itself an agent on a model).

I assume "harness" here just means the glue that feeds one model's output into that of another?

Definitely sounds expensive. Would it even be effective though? The more-privileged rings have to guard against [output from unprivileged rings] rather than [input to unprivileged rings]. Since the former is a function of the latter (in deeply unpredictable ways), it's hard for me to see how this fundamentally plugs the whole.

I'm very open to correction though, because this is not my area.

Re: Claude Cowork exfiltrates files

#339

Post by groby_b »

alienbaby wrote: post #129
groby_b wrote: post #63

Earlier quoted context omitted.

> We have all of the tools to prevent these agentic security vulnerabilities, We do? What is the tool to prevent prompt injection?

The best I've heard is rewriting prompts as summaries before forwarding them to the underlying ai, but has it's own obvious shortcomings, and it's still possible. If harder. To get injection to work

Alas, the summarizer... is vulnerable to prompt injection.
Post reply on HN