Observed Agent Sandbox Bypasses
voratiq.com
Observed Agent Sandbox Bypasses
1–10 of 52 posts
Re: Observed Agent Sandbox Bypasses
#2Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduced by the sandbox.
Re: Observed Agent Sandbox Bypasses
#3I am testing running agents in docker containers, with a script for managing different images for different use cases etc, and came across this: https://docs.docker.com/ai/sandboxes/
Has anyone given it a try?
Re: Observed Agent Sandbox Bypasses
#4I am testing running agents in docker containers, with a script for managing different images for different use cases etc, and came across this: https://docs.docker.com/ai/sandboxes/ Has anyone given it a try?
Would test it but it requires "Desktop". Immediate no... no reason to use that.
Re: Observed Agent Sandbox Bypasses
#5Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduc…
> These just sound like examples of friction introduced by the sandbox.
The whole idea of putting "agentic" LLMs inside a sandbox sounds like rubbing two pieces of sandpaper together in the hopes a house will magically build itself.
Re: Observed Agent Sandbox Bypasses
#6I am testing running agents in docker containers, with a script for managing different images for different use cases etc, and came across this: https://docs.docker.com/ai/sandboxes/ Has anyone given it a try?
> Has anyone given it a try?
Yes, I don't think this will persist caches & configs outside of the current dir, for example, the global npm/yarn/uv/cargo cache or even Claude/Codex/Gemini code config.
I ended up writing my own wrapper around Docker to do this. If interested, you can see the link in my previous comments. I don't want to post the same link again & again.
Re: Observed Agent Sandbox Bypasses
#7> The swap bypassed our policy because the deny rule was bound to a specific file path, not the file itself or the workspace root.
This policy is stupid. I mount the directory read inside the container to make it impossible to do it (except for a security leak in the container itself)
Re: Observed Agent Sandbox Bypasses
#8I am testing running agents in docker containers, with a script for managing different images for different use cases etc, and came across this: https://docs.docker.com/ai/sandboxes/ Has anyone given it a try?
I've been using container-use to do something like that: https://container-use.com/introduction
Re: Observed Agent Sandbox Bypasses
#9Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduc…
> These just sound like examples of friction introduced by the sandbox. The whole idea of putting "agentic" LLMs inside a sandbox sounds like rubbing two pieces of sandpaper together in the hopes a house will magically build itself.
That’s some good house-building sandpaper then.
Re: Observed Agent Sandbox Bypasses
#10Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduc…
> These just sound like examples of friction introduced by the sandbox. The whole idea of putting "agentic" LLMs inside a sandbox sounds like rubbing two pieces of sandpaper together in the hopes a house will magically build itself.
Trouble is it occasionally works