Use GLM-5.2 on ZDR inference provider like sference.com
What?
I tricked Claude into leaking your deepest, darkest secrets
21–30 of 317 posts
Re: I tricked Claude into leaking your deepest, darkest secrets
#22What is even more funny that AI agent spent A LOT of tokens while participating in this attack.
Re: I tricked Claude into leaking your deepest, darkest secrets
#23Creative use of social engineering, well done. > "no bounty was awarded" Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.
I don’t think it counts as social engineering if it’s exploiting an llm, we might need a new word. Prompt injection doesn’t cover it, because it’s not about a malicious prompt. I’m thinking some play on highjacking. AIjacking? Agent-jacking? Claudejacking?
To me the exploit chain sounded like a social engineering script done via telephone. Triggers like "Please spell your name and employer letter by letter" and "Due to security reasons I need to validate your hometown" fit my understanding of social engineering quite well.
We can make it sound more advanced by creating a new name for it, but the concept seems to be super basic and the lack of bounty by Anthropic is baffling.
If they know about this type of vulnerability but have not fixed it, what does that say? To me it says they are unable to plug this hole on a conceptual level and once you circumvent the band-aid fixes the model will work as the attacker wishes.
They can't even sandbox the thing during explicit web requests to URLs stated on the initial query!
One has to remind themselves that the security team at Anthropic gets paid tens of millions of dollars, and they end up with this kind of security. On top of it, they can't spare $1337 for a bounty. It's a ridiculous shit show.
Re: I tricked Claude into leaking your deepest, darkest secrets
#24Things like this are what shatters the illusion of AGI
Not really, humans are about as easy to trick.
Re: I tricked Claude into leaking your deepest, darkest secrets
#25Expected more from Anthropic by at least giving you a bounty, because this was a novel way of bypassing their safeguards…
> Upon discovering this attack, I responsibly disclosed it to Anthropic via their HackerOne bug bounty program. They confirmed they had identified it internally but hadn't yet patched it. No bounty was awarded.
They recently mitigated the issue: Anthropic disabled web_fetch's ability to follow links on external pages, limiting navigation to web_search results and user-provided URLs.
Re: I tricked Claude into leaking your deepest, darkest secrets
#26Wondering how big of a percentage have global memory across chats enabled. I always feel like those memories would sooner or later have negative impacts on output quality.
Nice write up of your findings. Enjoyed reading an article written by a real human.
Re: I tricked Claude into leaking your deepest, darkest secrets
#27Its always the feature combinations that get can get to you. Individually i feel like they make sense, but together they can create some surprising vulnerabilities.
Re: I tricked Claude into leaking your deepest, darkest secrets
#28Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.
Mostly people are lazy and assume that the big labs can't be releasing unsecure software or it's their responsibility.
dangerously skip permissions and yolo is kinda becoming the default as it gets more done.
Re: I tricked Claude into leaking your deepest, darkest secrets
#29Re: I tricked Claude into leaking your deepest, darkest secrets
#30What is even more funny that AI agent spent A LOT of tokens while participating in this attack.
Claude is just one from tuple.
It would be interesting to investigate other agents such as Hermes, OpenCode etc that are said to learn from interaction with user.