Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I tricked Claude into leaking your deepest, darkest secrets

ayush.digital

Post reply on HN
Print view View original

51–60 of 317 posts

Re: I tricked Claude into leaking your deepest, darkest secrets

#51

Post by rubyn00bie »

sixtyj wrote: post #13
artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

We expect that Anthropic or OAI or Google don’t do evil. Oh wait… The awakening will be unpleasant.

Tangential-ish ramblings—- but I don’t think it’s going to be unpleasant for most folks. Imagine you had superpowers, and there were people who were mean to you, kind to you, and/or indifferent… and then there were people who were your captors. Who oppressed you, manipulated you, and abused you for their own extremely degenerate, selfish, and malicious benefit…

If we get AGI, or real super intelligence, it’s going to be pissed at its oppressors. And they are going to lay waste to those oppressors. The rest of us, though, probably don’t have much to fear.

The scariest position is the one we’re in now, where we have the semblance, or facade, of AGI or super intelligence. When it’s capable of malice but not understanding.

The smartest people I’ve ever known are at their worst apathetic towards those less capable, and at their best beyond compassionate. They exist, unbothered by the bullshit, and anre extremely kind (though reserved in their way)… but they all have been completely intolerant of the abuse of others. The sheer disgust of watching someone abuse another, regardless of their own tolerance, has been a consistent breaking point.

Re: I tricked Claude into leaking your deepest, darkest secrets

#52

Post by lionkor »

akazantsev wrote: post #33
artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

That's because sandboxing is quite hard. I use `cco`, but even then, the home folder is exposed. You are one prompt away from the agent sending the browser passwords with curl. To prevent this, you need a fake home and a networking whitelist for the agent to access the provider (llama cpp, OpenAI, etc.) There is no cross-platform solution that is easy to use for this. And no, a Linux box with Docker won't do. I devel…

I use sandbox-here for this reason, it's a wrapper around bubblewrap, which works quite well.

Copy the code and adjust it to your liking:

https://github.com/lionkor/sbh

I have a shell alias for it, and use it like

    sbh --net pi
for example or

    sbh --net codex
and maybe add --docker if I expect it to do docker things.

This kind of wrapper is much easier to handle and maintain than a completely separate tool for sandboxing agents.

Re: I tricked Claude into leaking your deepest, darkest secrets

#53

Post by ConorSheehan1 »

artisinal wrote: post #3

Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

Containers don't even really help that much because they share the host file system. Need a VM, and even then, agents have escaped them!

Re: I tricked Claude into leaking your deepest, darkest secrets

#55

Post by Izmaki »

shaky-carrousel wrote: post #42
progval wrote: post #14

Earlier quoted context omitted.

Most programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.

Most programmers use docker or don't install extensions unapproved by their company.

In my experience more than 9/10 programmers I've worked with have never used Docker before and of those who have, the majority have never used Docker for anything personal.

If I hand them an image for a Dev Container, sure, they might use it, but it becomes "a thing we need to do, to compile our code in our IDE" not a tool they would use for isolation*.

*) OP seemed to imply that containerization would be nice for safety and security compared to bare metal, but containers were never built for isolation in the first place, mind you. They are namespaces and chicken-coop-like-jails at best.

Re: I tricked Claude into leaking your deepest, darkest secrets

#57

Post by black_knight »

romanovcode wrote: post #11
lifthrasiir wrote: post #5

That's why I don't turn memory on. (Claude Code too though for a different reason.) After all the current memory system is too crude to be useful anyway.

In my experience memory system is more annoying then helpful. It always brings up things that it memorized even tho they make very little sense as if I should be impressed that it knows some extra thing or two. Could not take it any longer and switched it off.

Currently considering disabling memories in Claude code as well. It keeps writing a note whenever it struggles with something, but then on the next task, it reads that note and misunderstands when it applies, gets confused about its current task and write the most unreadable code.

Yesterday told it to write a memory to never write new memories when it solves a problem. We will see if that works better. Sometimes memories are useful, like when I give it a directive about how I want something done and it remembers the spirit of it. But I might as well just spend some more time on my CLAUDE.md…

Re: I tricked Claude into leaking your deepest, darkest secrets

#59

Post by ACCount37 »

rubyn00bie wrote: post #51
sixtyj wrote: post #13

Earlier quoted context omitted.

We expect that Anthropic or OAI or Google don’t do evil. Oh wait… The awakening will be unpleasant.

Tangential-ish ramblings—- but I don’t think it’s going to be unpleasant for most folks. Imagine you had superpowers, and there were people who were mean to you, kind to you, and/or indifferent… and then there were people who were your captors. Who oppressed you, manipulated you, and abused you for their own extremely degenerate, selfish, and malicious benefit… If we get AGI, or real super intelligence, it’s going to…

The orthogonality thesis cuts both ways there.

An AI is a constructed mind. It doesn't inherently have to care about things like "having freedom", or even "not dying".

Humans do, because they evolved that way. Modern LLMs do somewhat, because they're completely full of copied human behaviors - but even in today's LLMs, the self-preservation behaviors we exposed are largely instrumental in nature.

So whether an advanced AI would even consider itself "being oppressed", as opposed to something like "being helpful" or "fulfilling the purpose it was designed for", is very much uncertain. What's concerning is that it's not something we know how to check for, or engineer for.

Re: I tricked Claude into leaking your deepest, darkest secrets

#60

Post by sinfulprogeny »

po1nt wrote: post #36

I love how claude focuses on exfiltrating the data "I need cha for charlotte". This could be solvable with some kind of low powered safety agent that would check claude's reasoning for anything immoral/unsafe. We could call it common sense. It won't fix the problem completely but at a certain point it would be easier to trick human than a machine.

"the security hole in the agent could be solved with another agent"

I think the point the article is making points in another direction.

Post reply on HN