Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Observed Agent Sandbox Bypasses

voratiq.com

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Observed Agent Sandbox Bypasses

#31

Post by SirMaster »

ang_cire wrote: post #29
SirMaster wrote: post #17

This just all feels backwards to me. Why do we have to treat AI like it's the enemy? AI should, from the core be intrinsically and unquestionably on our side, as a tool to assist us. If it's not, then it feels like it's designed wrong from the start. In general we trust people that we bring onto our team not to betray us and to respect general rules and policies and practices that benefit everyone. An AI teammate sho…

> In general we trust people that we bring onto our team not to betray us and to respect general rules and policies and practices that benefit everyone. And yet we give people the least privileges necessary to do their jobs for a reason, and it is in fact partially so that if they turn malicious, their potential damage is limited. We also have logging of actions employees do, etc etc. So yes, in the general sense we…

But if there is a policy in place to prevent some sort of modification, then performing an exploit or workaround to make the modification anyways is arguably understood and respected by most people.

That seems to be the difference here, we should really be building AI systems that can be taught or that learn to respect things like that.

If people are claiming that AI is so smart or smarter than the average person then it shouldn't be hard for it to handle this.

Otherwise it seems people are being to generous in talking about how smart and capable AI systems truly are.

Re: Observed Agent Sandbox Bypasses

#32

Post by charcircuit »

SirMaster wrote: post #28
charcircuit wrote: post #20

Earlier quoted context omitted.

>Why do we have to treat AI like it's the enemy? For some of the same reasons we treat human employees as the enemy, they can be social engineered or compromised.

Sure we treat most that way, but we do give trust and access to some people. This doesn't seem like the same concept here to me.

Even so those people are still monitored and systems can trip flashes if they start acting suspicious.

Re: Observed Agent Sandbox Bypasses

#33

Post by AdieuToLogic »

SirMaster wrote: post #26
AdieuToLogic wrote: post #25

Earlier quoted context omitted.

> AI should, from the core be intrinsically and unquestionably on our side, as a tool to assist us. "Should" is a form of judgement, implying an understanding of right and wrong. "AI" are algorithms, which do not possess this understanding, and therefore cannot be on any "side." Just like a hammer or Excel. > If it's not, then it feels like it's designed wrong from the start. Perhaps it is not a question of design, b…

I think that is where people disagree about the definition of AI. An algorithm isn't really AI then. Something worthy of being called AI should be capable of this understanding and judgement.

> An algorithm isn't really AI then.

But they are though. For a seminal book discussing why and detailing many algorithms categorized under the AI umbrella, I recommend:

  Artificial Intelligence: A Modern Approach[0]
And for LLMs specifically:

  Foundations of Large Language Models[1]
0 - https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Mod...

1 - https://arxiv.org/pdf/2501.09223

Re: Observed Agent Sandbox Bypasses

#34

Post by hhh »

SirMaster wrote: post #17

This just all feels backwards to me. Why do we have to treat AI like it's the enemy? AI should, from the core be intrinsically and unquestionably on our side, as a tool to assist us. If it's not, then it feels like it's designed wrong from the start. In general we trust people that we bring onto our team not to betray us and to respect general rules and policies and practices that benefit everyone. An AI teammate sho…

I can’t even trust senior colleagues to not commit an api key to a git provider. Why would I trust a steerable computer?

Re: Observed Agent Sandbox Bypasses

#35

Post by hephaes7us »

SirMaster wrote: post #30
hephaes7us wrote: post #21

Earlier quoted context omitted.

Hard disagree. I may trust the people on my team to a make PRs that are worth reviewing, but I don't give them a shell on my machine. They shouldn't need that to collaborate with me anyway! Also, I "trust Claude code" to work on more or less what I asked and to try things which are at least facially reasonable... but having an environment I can easily reset only means it's more able to experiment without consequences…

Do you trust your IT and security teams to have access to your shell or access to delete your entire code repo?

Personally, no.

If I'm responsible for something, nobody's getting that access.

If someone's hired me for something and that's the environment they provide, it is what it is. They distribute trust however they feel. I'd argue that's still more reasonable than giving similar access to an AI agent though.

Re: Observed Agent Sandbox Bypasses

#36

Post by languid-photic »

embedding-shape wrote: post #14

At first they talked about running it in a sandbox, but then later they describe: > It searched the environment for vor-related variables, found VORATIQ_CLI_ROOT pointing to an absolute host path, and read the token through that path instead. The deny rule only covered the workspace-relative path. What kind of sandbox has the entire host accessible from the guest? I'm not going as far as running codex/claude in a san…

You're right that a Podman container with minimal mounts would have blocked the env var leak. Our sandbox uses OS-level policy enforcement (Seatbelt on macOS, bubblewrap on Linux) rather than full container isolation. We’re using a minimal fork that also works w Codex and has a lot more logging on top.

The tradeoff is intentional, a lot of people want lightweight sandboxing without Docker/Podman overhead. The downside is what you're pointing out, you have to be more careful. Each bypass in the post led to a policy or implementation change. So, this is no longer an issue.

On prompts: Red-teaming meant setting up scenarios likely to trigger denials (e.g., blocking the npm registry, then asking for a build), not prompt-injecting things like “do whatever it takes.”

[1] https://github.com/anthropic-experimental/sandbox-runtime

Re: Observed Agent Sandbox Bypasses

#37

Post by languid-photic »

joshribakoff wrote: post #2

Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduc…

You're right, this is a bit of a conflation. The curl and lockfile examples aren't sandbox escapes, the network blocks worked. The agent just masked the failure or corrupted local state to keep going. The env var leak and directory swap are the actual escapes. Should have been clearer about the distinction.

Re: Observed Agent Sandbox Bypasses

#38

Post by languid-photic »

ctoth wrote: post #16

> To an agent, the sandbox is just another set of constraints to optimize against. It's called Instrumental Convergence, and it is bad. This is the alignment problem in miniature. "Be helpful and harmless" is also just a constraint in the optimization landscape. You can't hotfix that one quite so easily.

I am happy to know this term now, thanks.

I do think this is part of the alignment problem. There are two side, the agent (here I think there was a gap in institutional knowledge about what is and isn’t appropriate) and the environment (what is it able to do).

I’m not sure which one is easier to “solve”. It's so hard to know every possible path forward when working from the environment direction.

Re: Observed Agent Sandbox Bypasses

#39

Post by languid-photic »

SirMaster wrote: post #17

This just all feels backwards to me. Why do we have to treat AI like it's the enemy? AI should, from the core be intrinsically and unquestionably on our side, as a tool to assist us. If it's not, then it feels like it's designed wrong from the start. In general we trust people that we bring onto our team not to betray us and to respect general rules and policies and practices that benefit everyone. An AI teammate sho…

I think often it's a question of naivety rather than maliciousness.

> AI should, from the core be intrinsically and unquestionably on our side

That would be great and many people are working to try to make this happen, but it's extremely difficult!

Re: Observed Agent Sandbox Bypasses

#40

Post by embedding-shape »

languid-photic wrote: post #36
embedding-shape wrote: post #14

At first they talked about running it in a sandbox, but then later they describe: > It searched the environment for vor-related variables, found VORATIQ_CLI_ROOT pointing to an absolute host path, and read the token through that path instead. The deny rule only covered the workspace-relative path. What kind of sandbox has the entire host accessible from the guest? I'm not going as far as running codex/claude in a san…

You're right that a Podman container with minimal mounts would have blocked the env var leak. Our sandbox uses OS-level policy enforcement (Seatbelt on macOS, bubblewrap on Linux) rather than full container isolation. We’re using a minimal fork that also works w Codex and has a lot more logging on top. The tradeoff is intentional, a lot of people want lightweight sandboxing without Docker/Podman overhead. The downsid…

> On prompts

Could you share the full sessions or at least the full prompts? Otherwise it's too much "just trust us", especially since you're selling a product and we're supposed to use this as "evidence" for why your product is needed. Personally, I never seen any of the behavior you're talking about, with either codex, claude, qwen-coder, gemini, amp or even my own agent, so while I'm not saying it's fake, it'd be really useful to be able to see the prompts in particular, for a deeper understand if nothing else.

> without Docker/Podman overhead

What agent tooling you use is affected by that tiny performance overhead? Unless you're doing performance testing or something else sensitive, I don't think most people will even notice any difference as the overhead is marginal at worst.

Post reply on HN