Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

Post reply on HN
Print view View original

111–120 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#111

Post by vladms »

XorNot wrote: post #76
progval wrote: post #14

Earlier quoted context omitted.

Most programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.

It has never been easy to create separate users on Linux, certainly not for tasks where you need to switch between contexts. Docker was amongst the biggest steps forward on this in a long time.

I do not know since when (I am using it for couple of years), but in Arch, it is very simple to have two X sessions (by using "log out" > "switch user") for two different accounts, so switching it's just a Control-Alt-F7 away.

Additionally, one can make the main user part of the group of the development user, so that you can read/write easy in the development user account and it is even easier to share stuff.

Re: I tricked Claude into leaking your deepest, darkest secrets

#112

Post by dpacmittal »

akazantsev wrote: post #33
artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

That's because sandboxing is quite hard. I use `cco`, but even then, the home folder is exposed. You are one prompt away from the agent sending the browser passwords with curl. To prevent this, you need a fake home and a networking whitelist for the agent to access the provider (llama cpp, OpenAI, etc.) There is no cross-platform solution that is easy to use for this. And no, a Linux box with Docker won't do. I devel…

Use multipass by canonical. Works on macOS, windows, Linux.

Re: I tricked Claude into leaking your deepest, darkest secrets

#113

Post by ahk-dev »

artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

I think we're converging on two separate security models. One is capability minimization (filesystem, network, shell permissions). The other is context minimization. An agent that only has access to the files and memories relevant to the current task is much less dangerous even if it has the same tool permissions. We already optimize context for cost; I suspect we'll end up treating it as a security boundary too.

Re: I tricked Claude into leaking your deepest, darkest secrets

#115

Post by richrichardsson »

rlpb wrote: post #98
inopinatus wrote: post #61

Earlier quoted context omitted.

I’ve been recommending the use of consistent lies about name and date of birth to online systems since Eternal September began. Very few sites and systems justify accurate PII, and even for those I often still maintain dual accounts/profiles as necessary.

I like using a date of birth of 1 January. It's plausible but also hopefully suspicious how many people seem to be born that day if others do the same.

Same, my D.O.B. is 1/1/1970 for anything that doesn't justify having the real deal.

Re: I tricked Claude into leaking your deepest, darkest secrets

#116

Post by tmikaeld »

inopinatus wrote: post #61
port3000 wrote: post #47

My name in Claude is Silly Bean. I did it at first because it made me chuckle every time I opened Claude and it said 'Back again, Silly Bean?' But turns out I was playing 4D cybersecurity chess

I’ve been recommending the use of consistent lies about name and date of birth to online systems since Eternal September began. Very few sites and systems justify accurate PII, and even for those I often still maintain dual accounts/profiles as necessary.

That never works on Facebook though, because as soon as a ”friend” reports that ”I’m not me” then the account will be permanently banned. That also triggers for photos that’s not genuinely me, like a pet or drawing as portrait.

Re: I tricked Claude into leaking your deepest, darkest secrets

#118

Post by simonw »

kennywinker wrote: post #17
bflesch wrote: post #6

Creative use of social engineering, well done. > "no bounty was awarded" Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.

I don’t think it counts as social engineering if it’s exploiting an llm, we might need a new word. Prompt injection doesn’t cover it, because it’s not about a malicious prompt. I’m thinking some play on highjacking. AIjacking? Agent-jacking? Claudejacking?

I see the attack described here as a classic example of a prompt injection.

The attack works because malicious instructions were accessed (using the web_fetch tool) and concatenated together with the other agent input, in a way that then subverted the agent's behavior.

Re: I tricked Claude into leaking your deepest, darkest secrets

#119

Post by AtHeartEngineer »

inopinatus wrote: post #61
port3000 wrote: post #47

My name in Claude is Silly Bean. I did it at first because it made me chuckle every time I opened Claude and it said 'Back again, Silly Bean?' But turns out I was playing 4D cybersecurity chess

I’ve been recommending the use of consistent lies about name and date of birth to online systems since Eternal September began. Very few sites and systems justify accurate PII, and even for those I often still maintain dual accounts/profiles as necessary.

Jan 1st, 1984 every time

Re: I tricked Claude into leaking your deepest, darkest secrets

#120

Post by sph »

spaqin wrote: post #54
AndrewThrowaway wrote: post #22

What is even more funny that AI agent spent A LOT of tokens while participating in this attack.

The real winner of this 'attack' is Anthropic.

As long as you pay no attention to the man laughing all the way to the bank, Jensen Huang.
Post reply on HN