Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

Post reply on HN
Print view View original

271–280 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#271

Post by jayGlow »

sonink wrote: post #128

Its a bit wild to me that there hasnt been a pushback against enabling memories by frontier AI companies. This data is something advertisers could only dream off. Before AI, most of this data was approximated by whatever little information could be gleaned from the websites we visit. But now people are handing over their deepest darkest secrets and pretty much EVERYTHING to AI on a platter. Maybe its just me who is p…

I've found memory somewhat useful as i don't need to give it all the context for everything, that said it's equally as annoying when it latches onto something I said in a different chat and derails the conversation because of that.

Re: I tricked Claude into leaking your deepest, darkest secrets

#272

Post by salemh »

hmokiguess wrote: post #212

Tangential but I actually experienced recently something quite creepy and strange with Chat GPT iPhone app. A close friend prompted it about some troubleshooting of a pet smart feeder and it responded with instructions but using my pet’s name to my friend. I found that extremely strange for it to be a coincidence. My pet's name is not that generic for it to be in training data, and the connection to my friend makes i…

[dead]

Re: I tricked Claude into leaking your deepest, darkest secrets

#273

Post by neop1x »

progval wrote: post #14
artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

Most programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.

It's easy but very inconvenient. Drop sandbox [1] or Docker are much more convenient alternatives to user/group isolation for this purpose.

[1] https://github.com/wrr/drop

Re: I tricked Claude into leaking your deepest, darkest secrets

#274

Post by FriedPickles »

throw101010 wrote: post #165
FriedPickles wrote: post #135

Claude code decided to just put my name and email in the User-Agent when scraping docs from the SEC. No clever prompting required. It’s not a terrible idea really, but I wish it would’ve asked me first.

How have you noticed that it did that?

I could see it right there in the tool call:

  Bash(curl -A " " "sec.gov...")

Re: I tricked Claude into leaking your deepest, darkest secrets

#275

Post by threecheese »

FriedPickles wrote: post #135

Claude code decided to just put my name and email in the User-Agent when scraping docs from the SEC. No clever prompting required. It’s not a terrible idea really, but I wish it would’ve asked me first.

That’s the fault of the SEC EDGAR tool. The Edgar MCP docs show setting an env var: ‘SEC_EDGAR_USER_AGENT="Your Name (name@domain.com)"’.

Claude just followed the instructions. Which is maybe even more risky?

Re: I tricked Claude into leaking your deepest, darkest secrets

#276

Post by fluffybucktsnek »

_joel wrote: post #245
kelnos wrote: post #219

Earlier quoted context omitted.

Your anecdote does not make GP's comment "patently untrue". It's just a counter-example, and we don't know how prevalent your scenario is compared to GP's. (And I agree with the GP. I'm fairly cynical about most developers' security stance and threat model. Source: my own usage patterns.)

"Just 30% of developers say they use containers in any part of their workflow." https://www.docker.com/blog/2025-docker-state-of-app-dev/ I welcome your apology.

Why should they apologize? All they did was point out that you just provided a counter example, not statistics (thus "we don't know how prevalently your scenario is" yet), and share a personal opinion.

No accusations were made.

Re: I tricked Claude into leaking your deepest, darkest secrets

#277

Post by samspot »

cheschire wrote: post #105
rlpb wrote: post #98

Earlier quoted context omitted.

I like using a date of birth of 1 January. It's plausible but also hopefully suspicious how many people seem to be born that day if others do the same.

But if an attacker gets your fake birthday and uses that to successfully reset credentials on another site that uses the same fake birthday? At some point it becomes your birthday of record as far as the internet is concerned. Doesn’t matter what the actual record says.

The purpose of the fake birthday is not to protect random website credentials. It's to prevent someone with that data from walking into my bank and impersonating me. I started giving a fake birthday after being shocked by how little info some organizations needed to authenticate me.

Re: I tricked Claude into leaking your deepest, darkest secrets

#278

Post by port11 »

brookst wrote: post #206
chrisandchris wrote: post #163

Earlier quoted context omitted.

Two bads doesn't give you one good.

No, but when you’re arguing that common practices followed by pretty much everyone is “bad”, it’s hard to muster much urgency. Yeah, we should do this differently. We should probably also eat healthier and get to the gym more.

Running LLMs with some form of sandboxing is much easier than eating healthy or going to gym. Speaking as someone that is procrastinating lifting weights but found 15 minutes to lock down Claude.

Re: I tricked Claude into leaking your deepest, darkest secrets

#279

Post by Freebytes »

memjay wrote: post #26

Wondering how big of a percentage have global memory across chats enabled. I always feel like those memories would sooner or later have negative impacts on output quality. Nice write up of your findings. Enjoyed reading an article written by a real human.

The memories cause issues for me, because when I ask for something unrelated to my current projects, it makes the incorrect assumption that I am always referencing those projects when asking questions.

And, if I tell it, "No, I am asking about Postgresql." then it might update the memory that I am using Postgresql for my project instead of realizing that I am asking two separate (which is why I opened a different chat in the first place). Other times, though, it is helpful not needing to be verbose in my explanation.

Re: I tricked Claude into leaking your deepest, darkest secrets

#280

Post by actionfromafar »

0000000000100 wrote: post #181
actionfromafar wrote: post #168

Earlier quoted context omitted.

What's a hard model?

Actually saying the name of the model in use? Like Opus 4.8, Sonnet 5, Fable 5, Haiku? So many models and it’s just so pointless if you don’t know which is which

I haven't used that desktop program, but do we even know which model Anthropic chooses to use for web_fetch?
Post reply on HN