Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

261–270 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#261

Earlier quoted context omitted.

Separate user accounts are irrelevant when any one user has sudo and can therefore change binaries for everyone.

The point is to not give every user (especially the LLM user) sudo access.

Exactly. And you can always create a non-priv user and do the llm stuff using that user.

Re: I tricked Claude into leaking your deepest, darkest secrets

#262

My name in Claude is Silly Bean. I did it at first because it made me chuckle every time I opened Claude and it said 'Back again, Silly Bean?' But turns out I was playing 4D cybersecurity chess

But now it will match Silly Bean to port3000 and every comment you've made.

Re: I tricked Claude into leaking your deepest, darkest secrets

#264

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

It's convenience. Nothing beats it. Having an agent work alongside you with no restrictions gives instant gratification.

Agreed - I know it's poor security but damn does it work so well

I'm ok with the risk because I typically am pretty explicit about telling the agent what to do - I don't do the loops like "Do this until X" where the agent can make up its own workflow

When i tell it to add features, it doesn't try to do crazy things like installing packages or making up new paradigms - I usually tell it to do those things when I need to

Maybe this is security cope but at this point you'll have to pry unrestricted yolo mode from my cold dead hands. Maybe I'll change my mind when I pwn myself accidentally

I have a tough time with computer security because it's generally inconvenient and results in a worse developer and user experience

Re: I tricked Claude into leaking your deepest, darkest secrets

#265

My name in Claude is Silly Bean. I did it at first because it made me chuckle every time I opened Claude and it said 'Back again, Silly Bean?' But turns out I was playing 4D cybersecurity chess

But now it will match Silly Bean to port3000 and every comment you've made.

Oh...well I am a Silly Bean!

Re: I tricked Claude into leaking your deepest, darkest secrets

#266

Anthropic had to cut the legs off web_fetch to solve this issue, though. Now it can't page through any results on the target site to get the data you want.

Yeah, that’s not sustainable. Presumably they’ll come up with a guardrail (followed by an exploit, followed by another guardrail, repeat forever…).

Re: I tricked Claude into leaking your deepest, darkest secrets

#267
post #84

Earlier quoted context omitted.

He should clarify that "most" can be easily replaced by "all" as it was determined by statistical pool of whopping 1 person - himself. And also clarify that it's all lie. He just want to tell the anonymous crowd "look, I'm better than you".

You should also clarify that you pulled your statements out of your butt to look edgy. Everyone in every team I worked for the last ten years use docker. Docker is old tech. If you and your cavemen devs ignore what it is, that's your problem.

Sure, modern containerization is objectively good and should be used pretty much everywhere unless you have a strong reason not to, but the unfortunate reality is that it is nowhere near as universal as it should be.

Re: I tricked Claude into leaking your deepest, darkest secrets

#269
post #14

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

Most programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.

Is there a general workflow for this? I usually do pip under my user. I had not thought to do a su then do my venv and pip. Heck, are we at the point where we shouldn't even do that and everything should be done in a vm container?

Re: I tricked Claude into leaking your deepest, darkest secrets

#270

This is why I feel prompt injection is going to continue to be an issue. Fantastic that “Hi we are Cloudflare, give us your personal data” works. Either we stunt the models to the point where they are not useful, or we allow things like this to seep in and create one of the most insecure concepts the internet (and maybe tech as a whole) has ever seen: a robot that can be tricked.

I kinda can't get over the fact processed data can conversationally convince these LLMs to break security boundaries. Like, those malicious prompts are not illustrative analogies, but the actual attack strings. Absolutely crazy to me this tech is as widely used in automated interactions, but apparently can't be restricted on a logical, fundamental level. Is there really no functional understanding of the insides? No segmentation? Is it really just one fucking blob you have to convince to behave and pray someone else doesn't do a better job at it? Bonkers.
Post reply on HN