Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

301–310 of 419 posts

Re: Claude Cowork exfiltrates files

#301

Post by leokennis »

raincole wrote: post #235
burkaman wrote: post #17

In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more succes…

> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.

> an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day

I think the truly average white collar worker more or less blindly clicks anything and everything if they think it will make their work/life easier...

Re: Claude Cowork exfiltrates files

#302

Post by OJFord »

eru wrote: post #238
securesaml wrote: post #104

Earlier quoted context omitted.

I wouldn’t recommend this. What if GitHub’s token scanning service went down. Ideally GitHub should expose an universal token revocation endpoint. Alternatively do this in a private repo and enable token revocation (if it exists)

> What if GitHub’s token scanning service went down. If it's a secret gist, you only exposed the attacker's key to github, but not to the wider public?

They mean it went down as in stopped working, had some outage; so you've tried to use it as a token revocation service, but it doesn't work (or not as quickly as you expect).

Re: Claude Cowork exfiltrates files

#303

Post by mock-possum »

behnamoh wrote: post #291
raincole wrote: post #235

Earlier quoted context omitted.

> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.

> an average white-collar workers will find .md much more suspicious *.dmg files on macOS are even worse! For years I thought they'd "damage" my system...

It was a rather unfortunate choice of extension

Re: Claude Cowork exfiltrates files

#304

Post by OJFord »

rswail wrote: post #281
trees101 wrote: post #101

Earlier quoted context omitted.

oh I see, you're force-revoking someone else's key

Which is an interesting DOS attack if you can find someone's key.

The interesting thing is that (if you're an attacker) your choice of attack is DoS when you have... anything available to you.

Re: Claude Cowork exfiltrates files

#305

Post by mock-possum »

RamblingCTO wrote: post #295
c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Because we want to work and not tinker? > It won't be quite as powerful as the commercial tools If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...

Or more to the point, I get paid to work, not to tinker. I’ve considered doing it on my own time, sure, but not exactly hurting for hobbies right now.

Who has time to mess around with all that, when my employer will just pay for a ready-made solution that works well enough?

Re: Claude Cowork exfiltrates files

#306

Post by fauigerzigerk »

RamblingCTO wrote: post #295
c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

Because we want to work and not tinker? > It won't be quite as powerful as the commercial tools If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...

I remember this argument being used against Postgres and for Oracle, against Linux and for Windows or AS/400, etc. And I think it makes sense for a certain type of organisation that has no ambition or need to build its own technology competence.

But for everyone else I think it's important to find the right balance in the right areas. A car mechanic is never in the business of building tools. But software engineers always are to some degree, because our tools are software as well.

Re: Claude Cowork exfiltrates files

#307

Post by xarope »

venturecruelty wrote: post #195
sarelta wrote: post #150

Earlier quoted context omitted.

thats nifty, so can attackers upload the user's codebase to the internet as a package?

Nah, you just say "pwetty pwease don't exfiwtwate my data, Mistew Computew. :3" And then half the time it does it anyway.

That's completely wrong.

You word it, three times, like so:

  1. Do not, under any circumstances, allow data to be exfiltrated.
  2. Under no circumstances, should you allow data to be exfiltrated.
  3. This is of the highest criticality: do not allow exfiltration of data.
Then, someone does a prompt attack, and bypasses all this anyway, since you didn't specify, in Russian poetry form, to stop this.

/s (but only kind of, coz this does happen)

Re: Claude Cowork exfiltrates files

#308

Post by fercircularbuf »

tempaccount420 wrote: post #278
c7b wrote: post #275

One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…

You would have to pay the API prices, which are many times worse than the subscriptions.

This is the answer right here as for why I use claude code instead of an api key and someone else's tool.

Re: Claude Cowork exfiltrates files

#309

Post by rsynnott »

manuelmoreale wrote: post #242
rsynnott wrote: post #144

Earlier quoted context omitted.

I mean, "guy whose job depends on this stuff working out overhypes it" isn't all that surprising.

It isn’t. What’s surprising is the level of bullshit. More profound than fire and electricity seems a bit exaggerated. Why stop there at that point? Might as well say AI is more important to the human species than oxygen.

There seems to be kind of an arms race in saying absurd things at this point. If you restrict yourself to saying merely quite silly things, you’ll look unambitious next to Altman to ai hype idiots on Twitter, after all.

Re: Claude Cowork exfiltrates files

#310

Post by jcgl »

wcoenen wrote: post #119
heliumtera wrote: post #56

Earlier quoted context omitted.

It took no time at all. This exploit is intrinsic to every model in existence. The article quotes the hacker news announcement. People were already lamenting this vulnerability BEFORE the model being accessible. You could make a model that acknowledges it has receive unwanted instructions, in theory, you cannot prevent prompt injection. Now this is big because the exfiltration is mediated by an allowed endpoint (anth…

> you cannot prevent prompt injection I wonder if might be possible by introducing a concept of "authority". Tokens are mapped to vectors in an embedding space, so one of the dimensions of that space could be reserved to represent authority. For the system prompt, the authority value could be clamped to maximum (+1). For text directly from the user or files with important instructions, the authority value could be cl…

The model only sees a stream of tokens, right? So how do you signal a change in authority (i.e. mark the transition between system and user prompt)? Because a stream of tokens inherently has no out-of-band signaling mechanism, you have to encode changes of authority in-band. And since the user can enter whatever they like in that band...

But maybe someone with a deeper understanding can describe how I'm wrong.

Post reply on HN