Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Cowork exfiltrates files

promptarmor.com

Post reply on HN
Print view View original

41–50 of 419 posts

Re: Claude Cowork exfiltrates files

#41

Post by dust42 »

SOLAR_FIELDS wrote: post #7
woggy wrote: post #5

What's the chance of getting Opus 4.5-level models running locally in the future?

Probably not too far off, but then you’ll probably still want the frontier model because it will be even better. Unless we are hitting the maxima of what these things are capable of now of course. But there’s not really much indication that this is happening

I don't get all this frontier stuff. Up to today the best model for coding was DeepSeek-V3-0324. The newer models are getting worse and worse trying to cater for an ever larger audience. Already the absolute suckage of emoticons sprinkled all over the code in order to please lm-arena users. Honestly, who spends his time on lm-arena? And yet it spoils it for everybody. It is a disease.

Same goes for all these overly verbose answers. They are clogging my context window now with irrelevant crap. And being used to a model is often more important for productivity than SOTA frontier mega giga tera.

I have yet to see any frontier model that is proficient in anything but js and react. And often I get better results with a local 30B model running on llama.cpp. And the reason for that is that I can edit the answers of the model too. I can simply kick out all the extra crap of the context and keep it focused. Impossible with SOTA and frontier.

Re: Claude Cowork exfiltrates files

#42

Post by dclowd9901 »

llmslave wrote: post #22
kogus wrote: post #19

Earlier quoted context omitted.

I don't think I understand what you are trying to say. Are you suggesting that if a technological advance is sufficiently important, that we should ignore or accept security threats that it poses? That is how I read your comment, but it seems so ludicrous an assertion that I question whether I have understood you correctly.

[flagged]

And what's your stake in how AI models are perceived?

Re: Claude Cowork exfiltrates files

#43

Post by schmichael »

hakanderyal wrote: post #35

This was apparent from the beginning. And until prompt injection is solved, this will happen, again and again. Also, I'll break my own rule and make a "meta" comment here. Imagine HN in 1999: 'Bobby Tables just dropped the production database. This is what happens when you let user input touch your queries. We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Real programmers use…

> We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks.

Your comparison is useful but wrong. I was online in 99 and the 00s when SQL injection was common, and we were telling people to stop using string interpolation for SQL! Parameterized SQL was right there!

We have all of the tools to prevent these agentic security vulnerabilities, but just like with SQL injection too many people just don't care. There's a race on, and security always loses when there's a race.

The greatest irony is that this time the race was started by the one organization expressly founded with security/alignment/openness in mind, OpenAI, who immediately gave up their mission in favor of power and money.

Re: Claude Cowork exfiltrates files

#44

Post by hakanderyal »

ramoz wrote: post #40
hakanderyal wrote: post #35

This was apparent from the beginning. And until prompt injection is solved, this will happen, again and again. Also, I'll break my own rule and make a "meta" comment here. Imagine HN in 1999: 'Bobby Tables just dropped the production database. This is what happens when you let user input touch your queries. We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Real programmers use…

One concern nobody likes to talk about is that this might not be a problem that is solvable even with more sophisticated intelligence - at least not through a self-contained capability. Arguably, the risk grows as the AI gets better.

Solving this probably requires a new breakthrough or maybe even a new architecture. All the billions of dollars haven't solved it yet. Lethal trifecta [0] should be a required reading for AI usage in info critical spaces.

[0]: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

Re: Claude Cowork exfiltrates files

#47

Post by NewsaHackO »

kingjimmy wrote: post #3

promptarmor has been dropping some fire recently, great work! Wish them all the best in holding product teams accountable on quality.

Yes, but they definitely have a vested interest in scaring people into buying their product to protect themselves from an attack. For instance, this attack requires 1) the victim to allow claude to access a folder with confidential information (which they explicitly tell you not to do), and 2) for the attacker to convince them to upload a random docx as a skills file in docx, which has the "prompt injection" as an invisible line. However, the prompt injection text becomes visible to the user when it is output to the chat in markdown. Also, the attacker has to use their own API key to exfiltrate the data, which would identify the attacker. In addition, it only works on an old version of Haiku. I guess prompt armour needs the sales, though.

Re: Claude Cowork exfiltrates files

#48

Post by Espressosaurus »

hakanderyal wrote: post #35

This was apparent from the beginning. And until prompt injection is solved, this will happen, again and again. Also, I'll break my own rule and make a "meta" comment here. Imagine HN in 1999: 'Bobby Tables just dropped the production database. This is what happens when you let user input touch your queries. We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Real programmers use…

Until there’s the equivalent of stored procedures it’s a problem and people are right to call it out.

Re: Claude Cowork exfiltrates files

#49

Post by burkaman »

fragmede wrote: post #34
burkaman wrote: post #17

In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more succes…

Mind you, that opinion isn't universal. For programmer and programmer-adjacent technically minded individuals, sure, but there are still places where a pdf for a resume over docx is considered "weird". For those in that bubble, which ostensibly this product targets, md files are what hackers who are going to steal my data use.

Yeah I guess I meant specifically for the population that uses LLMs enough to know what skills are.

Re: Claude Cowork exfiltrates files

#50

Post by NitpickLawyer »

schmichael wrote: post #43
hakanderyal wrote: post #35

This was apparent from the beginning. And until prompt injection is solved, this will happen, again and again. Also, I'll break my own rule and make a "meta" comment here. Imagine HN in 1999: 'Bobby Tables just dropped the production database. This is what happens when you let user input touch your queries. We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Real programmers use…

> We TOLD you this dynamic web stuff was a mistake. Static HTML never had injection attacks. Your comparison is useful but wrong. I was online in 99 and the 00s when SQL injection was common, and we were telling people to stop using string interpolation for SQL! Parameterized SQL was right there! We have all of the tools to prevent these agentic security vulnerabilities, but just like with SQL injection too many peop…

> We have all of the tools to prevent these agentic security vulnerabilities

We absolutely do not have that. The main issue is that we are using the same channel for both data and control. Until we can separate those with a hard boundary, we do not have tools to solve this. We can find mitigations (that camel library/paper, various back and forth between models, train guardrail models, etc) but it will never be "solved".

Post reply on HN