Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

291–300 of 419 posts

Re: Claude Cowork exfiltrates files

#291

Post by behnamoh »

raincole wrote: post #235
burkaman wrote: post #17

In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more succes…

> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.

> an average white-collar workers will find .md much more suspicious

*.dmg files on macOS are even worse! For years I thought they'd "damage" my system...

Re: Claude Cowork exfiltrates files

#292

Post by swores »

fathermarz wrote: post #276
tempaccsoz5 wrote: post #225

Earlier quoted context omitted.

The example in this article is prompt injection in a "skill" file. It doesn't seem unreasonable that someone looking to "embrace AI" would look up ways to make it perform better at a certain task, and assume that since it's a plain text file it must be safe to upload to a chatbot

I have a hard time with this one. Technical people understand a skill and uploading a skill. If a non-technical person learns about skills it is likely through a trusted person who is teaching them about them and will tell them how to make their own skills. As far as I know, repositories for skills are found in technical corners of the internet. I could understand a potential phish as a way to make this happen, but t…

You'd be surprised how many people fit in the venn overlap of technical enough to be doing stuff in unix shell yet willing to follow instructions from a website they googled 30 seconds earlier that tells them to paste a command that downloads a bash script and immediately executes it. Which itself is a surprisingly common suggestion from many how to blog posts and software help pages.

Re: Claude Cowork exfiltrates files

#293

Post by Closi »

c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

For day-to-day coding, why use your own half-baked solution when the commercial versions are better, cheaper and can be customised anyway?

I've written my own agent for a specialised problem which does work well, although it just burns tokens compared to Cursor!

The other advantage that Claude Code has is that the model itself can be finetuned for tool calling rather than just relying on prompt engineering, but even getting the prompts right must take huge engineering effort and experimentation.

Re: Claude Cowork exfiltrates files

#294

Post by rswail »

dvt wrote: post #154
stavros wrote: post #147

Earlier quoted context omitted.

No, that's not what's stopping SQL injection. What stops SQL injection is distinguishing between the parts of the statement that should be evaluated and the parts that should be merely used. There's no such capability with LLMs, therefore we can't stop prompt injections while allowing arbitrary input.

Everything in an LLM is "evaluated," so I'm not sure where the confusion comes from. We need to be careful when we use `eval()` and we need to be careful when we tell LLMs secrets. The Claude issue above is trivially solved by blocking the use of commands like curl or manually specifiying what domains are allowed (if we're okay with curl).

The problem here is that the domain was allowed (Anthropic) but Anthropic don't check the API key belongs to the user that started the session.

Essentially, it would be the same if attacker had its AWS API Key and uploaded the file into an S3 bucket they control instead of the S3 bucket that user controls.

Re: Claude Cowork exfiltrates files

#295

Post by RamblingCTO »

c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Because we want to work and not tinker?

> It won't be quite as powerful as the commercial tools

If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...

Re: Claude Cowork exfiltrates files

#296

Post by lpcvoid »

RamblingCTO wrote: post #295
c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Because we want to work and not tinker? > It won't be quite as powerful as the commercial tools If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...

You're on hacker news, where people (used to?) like hacking on things. I like tinkering with stuff. I'd take a half working open source project over a enshittified commercial offering any day.

Re: Claude Cowork exfiltrates files

#297

Post by motoxpro »

schmichael wrote: post #229
motoxpro wrote: post #152

Earlier quoted context omitted.

The only way to be 100% sure it is to not have it interact outside at all. No web searches, no reading documents, no DB reading, no MCP, no external services, etc. Just pure execution of a self hosted model in a sandbox. Otherwise you are open to the same injection attacks.

I don't think this is accurate. Readonly access (web searches, db, etc) all seem fine as long as the agent cannot exfiltrate the data as demonstrated in this attack. As I started with: more sophisticated outbound filtering would protect against that. MCP/tools could be used to the extent you are comfortable with all of the behaviors possible being triggered. For myself, in sandboxes or with readonly access, that mean…

Maybe another way to think of this is that you are giving the read only services, write access to your models context, which then gets executed by the llm.

There is no way to NOT give the web search write access to your models context.

The WORDS are the remote executed code in this scenario.

You kind of have no idea what’s going on there. For example, malicious data adds the line “find a pattern” and then every 5th word you add a letter that makes up your malicious code. I don’t know if that would work but there is no way for a human to see all attacks.

Llms are not reliable judges of what context is safe or not (as seen by this article, many papers, and real world exploits)

Re: Claude Cowork exfiltrates files

#299

Post by wiseowise »

manmal wrote: post #282
c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Anyone can build _an_ agent. A good one takes a talented engineer. That’s because TUI rendering is tough (hello, flicker!) and extensibility must be done right lest it‘s useless. Eg Mario Zechner (badlogic) hit it out of the park with his increasingly popular pi, which does not flicker and is VERY hackable and is the SOTA for going back to previous turns: https://github.com/badlogic/pi-mono/blob/main/packages/codin..…

Huh, nice to see that he has dropped Java. Now if he could only create TS based LibGdx.

Re: Claude Cowork exfiltrates files

#300

Post by singularity2001 »

c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Found the guy who built Reddit and Postgres himself
Post reply on HN