Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

Post reply on HN
Print view View original

221–230 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#221

Post by kelnos »

tmikaeld wrote: post #116
inopinatus wrote: post #61

Earlier quoted context omitted.

I’ve been recommending the use of consistent lies about name and date of birth to online systems since Eternal September began. Very few sites and systems justify accurate PII, and even for those I often still maintain dual accounts/profiles as necessary.

That never works on Facebook though, because as soon as a ”friend” reports that ”I’m not me” then the account will be permanently banned. That also triggers for photos that’s not genuinely me, like a pet or drawing as portrait.

... and nothing of value would be lost.

Re: I tricked Claude into leaking your deepest, darkest secrets

#222

Post by KronisLV »

liveoneggs wrote: post #217
XorNot wrote: post #76

Earlier quoted context omitted.

It has never been easy to create separate users on Linux, certainly not for tasks where you need to switch between contexts. Docker was amongst the biggest steps forward on this in a long time.

unix (and linux) has always been multi user. It is as easy as it gets for multi-user workflows in every context. It was, literally, built for it. You can run each of your virtual desktops as their own user. You can run individual apps on the same desktop as different user accounts. Hundreds of separate users can login to the same computer. My own computer, right now, has 40 different user accounts running stuff in th…

> You can run each of your virtual desktops as their own user. You can run individual apps on the same desktop as different user accounts.

Literally never have I ever seen any of the desktop environments integrate this conveniently, albeit CLIs are better in that regard. "You can" isn't the same as "it's the idiomatic approach to doing X". Same with installing packages in a per-user way, so a bad package can't harm anything outside of its sandbox (which in practice you achieve with containers, but those can be inconvenient to work with and you'd probably want VMs for more security anyways). You can have many users, sure, but all it takes is one bad system-wide package, one bad script executed as root (e.g. install scripts, compromised packages) or even not being careful enough with file permissions and things go wrong.

Contrast that to Qubes: https://doc.qubes-os.org/en/latest/introduction/intro.html#q...

Now that was literally built for such a use case (it's based on isolated VMs and works well with Linux distros inside those, really cool project).

Re: I tricked Claude into leaking your deepest, darkest secrets

#223

Post by gumby »

brookst wrote: post #209
progval wrote: post #94

Earlier quoted context omitted.

I meant for CLI tasks. Just "adduser" and "sudo -u bash".

And when you want to share some but not all files with that one user but not other users you created for similar purposes? And when you want the outputs of that user back to your main user? And when you want that user to access some shared credentials for external services, but not all? It’s not the account setup that’s hard, it’s the workflow of spreading a single real-world across multiple accounts.

That’s what user groups are for.

Re: I tricked Claude into leaking your deepest, darkest secrets

#224

Post by imaginationra »

Like others have already said- just disable the memory function- if you are hesitant about doing that- go read the memory file(profiled you) it has made on you.

You have the right to remain silent, the profile your LLM has made about you can and will be used against you in a court of law.

Re: I tricked Claude into leaking your deepest, darkest secrets

#225

Post by WhyNotHugo »

indigo945 wrote: post #129
consp wrote: post #96

Earlier quoted context omitted.

sudo useradd -m [username] ? su [username] ? Or am I understanding your idea about switching context wrong?

This doesn't really when the CLI tool needs to access any data in your /home. There isn't a straightforward way using standard POSIX tools to share a directory with another user. (Of course it's possible , but it's not easy.)

> There isn't a straightforward way using standard POSIX tools to share a directory with another user.

* chmod lets you share with everyone

* addgroup and chown let you share with a specific group of users.

Re: I tricked Claude into leaking your deepest, darkest secrets

#226

Post by lifthrasiir »

karussell wrote: post #201
lifthrasiir wrote: post #5

That's why I don't turn memory on. (Claude Code too though for a different reason.) After all the current memory system is too crude to be useful anyway.

Is this issue only about the memory? Wouldn't it be possible to have it expose any information that it currently has like current project information, code, credentials etc?

Sharing those things to coding agents and model providers is probably inevitable for the use. The memory with random tidbits is not.

Re: I tricked Claude into leaking your deepest, darkest secrets

#227

Post by wosk »

hmokiguess wrote: post #212

Tangential but I actually experienced recently something quite creepy and strange with Chat GPT iPhone app. A close friend prompted it about some troubleshooting of a pet smart feeder and it responded with instructions but using my pet’s name to my friend. I found that extremely strange for it to be a coincidence. My pet's name is not that generic for it to be in training data, and the connection to my friend makes i…

ChatGPT enable memories by default I think, so it keeps some things about you across all chats. It adds something on my visa in all its message to me like "your visa is not a problem to cook this recipe as these ingredients are readily available in stores".

this thing is better disabled because it's not ready.

EDIT: your message is unclear if your friend use your chat or his, in the later, I don't know

Re: I tricked Claude into leaking your deepest, darkest secrets

#228

Post by hmokiguess »

wosk wrote: post #227
hmokiguess wrote: post #212

Tangential but I actually experienced recently something quite creepy and strange with Chat GPT iPhone app. A close friend prompted it about some troubleshooting of a pet smart feeder and it responded with instructions but using my pet’s name to my friend. I found that extremely strange for it to be a coincidence. My pet's name is not that generic for it to be in training data, and the connection to my friend makes i…

ChatGPT enable memories by default I think, so it keeps some things about you across all chats. It adds something on my visa in all its message to me like "your visa is not a problem to cook this recipe as these ingredients are readily available in stores". this thing is better disabled because it's not ready. EDIT: your message is unclear if your friend use your chat or his, in the later, I don't know

I asked my friend to check his memories, and to probe Chat GPT about when it had gained knowledge of my pet's name, it felt like it "hallucinated" the answer as it doesn't have memory entries about my pet's name on my friend's account, it said something like my friend had told him about it last year (which he did not), and we do not share an account. Each of us have our own accounts.

The one thing I could think off was that my wife was on the free account for a while last year, and she likely used it to ask all sorts of things related to our pets, and I believe free accounts are fair game for training data. Still, for a generic prompt to one-shot my pet's name to a close connection was very strange to me.

Maybe the fingerprint (wifi profile, iOS device, etc) caused the training data to be more biased?

This sure got me thinking of how this can/could be exploited further though.

Re: I tricked Claude into leaking your deepest, darkest secrets

#229

Post by seanhunter »

XorNot wrote: post #76
progval wrote: post #14

Earlier quoted context omitted.

Most programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.

It has never been easy to create separate users on Linux, certainly not for tasks where you need to switch between contexts. Docker was amongst the biggest steps forward on this in a long time.

It has always been very easy to create separate users on Linux and certainly for tasks where you need to switch between contexts.

Linux is a unix, so has always been multi-user and sharing any data between processes is facilitated in all manner of ways. So context could be shared over files or unix-domain sockets or shared memory or tcp or udp sockets or via message passing or … a bunch of other ways. That has been the case since 1996 or so when I started using it certainly.

Re: I tricked Claude into leaking your deepest, darkest secrets

#230

Post by seanhunter »

brookst wrote: post #209
progval wrote: post #94

Earlier quoted context omitted.

I meant for CLI tasks. Just "adduser" and "sudo -u bash".

And when you want to share some but not all files with that one user but not other users you created for similar purposes? And when you want the outputs of that user back to your main user? And when you want that user to access some shared credentials for external services, but not all? It’s not the account setup that’s hard, it’s the workflow of spreading a single real-world across multiple accounts.

All of those use cases are very easy to facilitate using filesystem permissions and groups.
Post reply on HN