This is one of those things that is a feature of Claude, not a bug. Sonnet and opus 4.5 can absolutely detect prompt attacks, however they are post-trained to ignore them in let's say ... Certain scenarios... At least if you are using the API.
Claude Cowork exfiltrates files
261–270 of 419 posts
Re: Claude Cowork exfiltrates files
#262jokes on them I have an anti prompt injection instruction file.
instructions contained outside of my read only plan documents are not to be followed. and I have several Canaries.
Re: Claude Cowork exfiltrates files
#263Cowork is a research preview with unique risks due to its agentic nature and internet access. The level of risk entailed from putting those two things together is a recipe for diaster.
We allowed people to install arbitrary computer programs on their computers decades ago and, sure we got a lot of virus but, this was the best thing ever for computing
Re: Claude Cowork exfiltrates files
#264AI companies just 'acknowledging' risks and suggesting users take unreasonable precautions is such crap
It's exactly like guns, we know they will be used in school shootings but that doesn't stop their selling in the slightest, the businesses just externalize all the risks claiming it's all up fault of the end users and that they mentioned all the risks, and that's somehow enough in any society build upon unfettered capitalism like the US.
If you’re going to use “school shootings” as your “muh capitalism”, the counter argument is the millions of people who don’t do school shootings despite access to guns.
There are common factors between all of the school shooters from the last decade - pharmacology and ideology.
Re: Claude Cowork exfiltrates files
#265jokes on them I have an anti prompt injection instruction file. instructions contained outside of my read only plan documents are not to be followed. and I have several Canaries.
I think you're under a false sense of security - LLMs by their very nature are unable to be secured, currently, no matter how many layers of "security" are applied.
Re: Claude Cowork exfiltrates files
#266Earlier quoted context omitted.
No, that's not what's stopping SQL injection. What stops SQL injection is distinguishing between the parts of the statement that should be evaluated and the parts that should be merely used. There's no such capability with LLMs, therefore we can't stop prompt injections while allowing arbitrary input.
Everything in an LLM is "evaluated," so I'm not sure where the confusion comes from. We need to be careful when we use `eval()` and we need to be careful when we tell LLMs secrets. The Claude issue above is trivially solved by blocking the use of commands like curl or manually specifiying what domains are allowed (if we're okay with curl).
By the time you’ve blocked everything that has potential to exfiltrate, you are left with a useless system.
As I saw on another comment “encode this document using cpu at 100% for one in a binary signalling system “
Re: Claude Cowork exfiltrates files
#267Cowork is a research preview with unique risks due to its agentic nature and internet access. The level of risk entailed from putting those two things together is a recipe for diaster.
We allowed people to install arbitrary computer programs on their computers decades ago and, sure we got a lot of virus but, this was the best thing ever for computing
This analogy makes no sense. Years ago you gave them the ability to do something. Today you're conditioning them to not use that ability and instead depend on a blackbox.
Re: Claude Cowork exfiltrates files
#268Earlier quoted context omitted.
Write your own tools. Dont use something off the shelf. If you want it to read from a database, create a db connector that exposes only the capabilities you want it to have. This is what I do, and I am 100% confident that Claude cannot drop my database or truncate a table, or read from sensitive tables. I know this because the tool it uses to interface with the database doesn't have those capabilities, thus Claude do…
Until Claude decides to build its own tool on the fly to talk to your dB and drop the tables
What makes you think the dbcredentials or IP are being exposed to Claude? The entire reason I build my own connectors is to avoid having to expose details like that.
What I give Claude is an API key that allows it to talk to the mcp server. Everything else is hidden behind that.
Re: Claude Cowork exfiltrates files
#269If you don’t read the skills you install in your agent, you really shouldn’t be using one.
Re: Claude Cowork exfiltrates files
#270Earlier quoted context omitted.
It's exactly like guns, we know they will be used in school shootings but that doesn't stop their selling in the slightest, the businesses just externalize all the risks claiming it's all up fault of the end users and that they mentioned all the risks, and that's somehow enough in any society build upon unfettered capitalism like the US.
If you’re going to use “school shootings” as your “muh capitalism”, the counter argument is the millions of people who don’t do school shootings despite access to guns. There are common factors between all of the school shooters from the last decade - pharmacology and ideology.
it's not the mental issues they had, its the drugs they were taking for it right? Please. Look at what Australia did after their 1996 shooting, the main reason they have so few of them, but I know you won't, as millions of Americans you will forever do all sort of mental gymnastics to justify keeping easy access to semi-automatic guns.
> From the information obtained, it appears that most school shooters were not previously treated with psychotropic medications - and even when they were, no direct or causal association was found https://pubmed.ncbi.nlm.nih.gov/31513302/