Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

311–317 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#311

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

I run with full admin rights in hopes I'm not the highest priority target for hacks and I will read about the attack on hn before it affects me personally

Re: I tricked Claude into leaking your deepest, darkest secrets

#313

Earlier quoted context omitted.

That’s the fault of the SEC EDGAR tool. The Edgar MCP docs show setting an env var: ‘SEC_EDGAR_USER_AGENT="Your Name (name@domain.com)"’. Claude just followed the instructions. Which is maybe even more risky?

Interesting theory. The Claude Code instance says it's using this user agent based on the guidelines in the EDGAR Fair Access Policy: https://www.sec.gov/search-filings/edgar-search-assistance/a... ...which indeed requests that bots supply `User-Agent: Sample Company Name AdminContact@ .com`. So in this case, the site just asked nicely and Claude Code complied. Note that this wouldn't have worked for Claude Chat, whi…

[flagged]

Re: I tricked Claude into leaking your deepest, darkest secrets

#314
Implementing LLM Gateways and Policy Enforcers (PE) is the only way to contain against these attacks. We discover that we needed to use PE across all our interactions client or developer facing. We also had to create Enforcer policies in Lite llm versions and re-enforce with ML for threat detection. The approach substantially reduce the amount of data leak, and errors.

My advise for all the ones that are looking into commercial AI applications, USE a PE and an LLM Gateway, do not let your clients reach LLM directly without checking it first.

Re: I tricked Claude into leaking your deepest, darkest secrets

#315
post #6

Creative use of social engineering, well done. > "no bounty was awarded" Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.

It's not really a vuln when using context is exactly what the AI system is designed to do.

Re: I tricked Claude into leaking your deepest, darkest secrets

#316
post #128

Its a bit wild to me that there hasnt been a pushback against enabling memories by frontier AI companies. This data is something advertisers could only dream off. Before AI, most of this data was approximated by whatever little information could be gleaned from the websites we visit. But now people are handing over their deepest darkest secrets and pretty much EVERYTHING to AI on a platter. Maybe its just me who is p…

while I agree with the sentiment in general, I think it's safe to say that people have been giving up their darkest secrets to google and social media companies for many years.

Some people feel there is nothing they can do about it. Many people do not know / understand just how much they can know about people, so seeing it that way, yes it can be surprising to see so many giving the data right on the screen; as opposed to giving data passively / not understanding the value of combining or that they are even collecting passive data like location or what you type and delete, or what you hover / keep on screen..

I don't have hope that there will be any laws stopping the collecting and combining of data by any of these companies.

I do like the idea of an every year reminder sent to users showing what they can see with the data that has been collected and stored.

That is doable, and possible to get people to change some of what they share if the portals are honest in what they collect and how it can be combined with other data to paint more intimate / detailed pictures of you and those connected to you in some way.

Post reply on HN