Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go hard on agents, not on your filesystem

jai.scs.stanford.edu

Post reply on HN
Print view View original

241–250 of 374 posts

Re: Go hard on agents, not on your filesystem

#241

Post by cyanydeez »

reader_1000 wrote: post #228
AnotherGoodName wrote: post #15

Add this to .claude/settings.json: { "sandbox": { "enabled": true, "filesystem": { "allowRead": ["."], "denyRead": ["~/"], "allowWrite": ["."], "denyWrite": ["/"] } } } You can change the read part if you're ok with it reading outside. This feature was only added 10 days ago fwiw but it's great and pretty much this.

For some reason, this made everything worse for me. Now claude constantly tries to access my home folder instead of current directory. Obviously this is not still good enough. Also Claude keeps dismissing my instructions on not to read my home directory and use current directory. Weird.

The problem with all these LLM instructed security features is the `codeword` poison probability.

The way LLMs process instructions isn't intelligence as we humans know it, but as the probability that an instruction will lead to an output.

When you don't mention $HOME in the context, the probability that it will do anything with $HOME remains low. However, if you mention it in the context, the probability suddenly increases.

No amount of additional context will have the same probability of never having poisoned the context by mentioning it. Mentioning $HOME brings in a complete change in probabilities.

These coding harnesses aren't enough to secure a safe operating environment because they inject poison context that _NO_ amount of textual context can rewire.

You just lost the game.

Re: Go hard on agents, not on your filesystem

#242

Post by puttycat »

nunez wrote: post #127
puttycat wrote: post #75

I am still amazed that people so easily accepted installing these agents on private machines. We've been securing our systems in all ways possible for decades and then one day just said: oh hello unpredictable, unreliable, Turing-complete software that can exfiltrate and corrupt data in infinite unknown ways -- here's the keys, go wild.

Tbf, Docker had a similar start. “Just download this image from Docker Hub! What can go wrong?!” Industry caught on quick though.

True, but the Docker attack surface is limited to a malicious actor distributing malicious images. (Bad enough in itself, I agree.)

Unreliable, unpredictable AI agents (and their parent companies) with system-wide permissions are a new kind of threat IMO.

Re: Go hard on agents, not on your filesystem

#243

Post by 100721 »

mcv wrote: post #197
ethanwillis wrote: post #190

Earlier quoted context omitted.

What we need is a capabilities based security system. It could write all the python, asm, whatever it wants and it wouldn't matter at all if it was never given a reference to use something it shouldn't.

Isn't this already possible? Give it its own user account with write access to the project directory and either read access or no access outside it.

I have been putting my agents on their own, restricted OS-level user accounts for a while. It works really well for everything I do.

Admittedly, there’s a little more friction and agent confusion sometimes with this setup, but it’s worth the benefit of having zero worries about permissions and security.

Re: Go hard on agents, not on your filesystem

#244

Post by esperent »

lxgr wrote: post #225
esperent wrote: post #73

Earlier quoted context omitted.

I added a hook to disable rm, find - delete, and a few of the other more obvious destructive ops. It sends Claude a strongly worded message: "STOP IMMEDIATELY. DO NOT TRY TO FIND WORKAROUNDS...". It works well. Git rm is still allowed.

It works well so far, for you . Are you confident it would still work against sophisticated prompt injection attacks that override your "strongly worded message"? Strongly worded signs can be great for safety (actual mechanisms preventing undesirable actions from being taken are still much better), but are essentially meaningless for security.

I mean, that's like saying are you sure that your antivirus would prevent every possible virus? Are you sure that you haven't made some mistake in your dev box setup that would allow a hacker to compromise it? What if a thief broke i to your house and stole your laptop? That's happened to me before, much more annoying to recover from that an accidental rm rf.

I do my best to keep off site back ups and don't worry about what I can't control.

Re: Go hard on agents, not on your filesystem

#246

Post by KronisLV »

eru wrote: post #181
KronisLV wrote: post #171

Earlier quoted context omitted.

Maybe Anthropic (or some collection of the large AI orgs, like OpenAI and Anthropic and Google coming together) should apply patches on top of (or fork altogether) the coreutils and whatever you normally get in a userland - a bit like what you get in Git Bash on Windows, just with: 1) more guardrails in place 2) maybe more useful error messages that would help LLMs 3) no friction with needing to get any patches upstr…

So for me, it's really, really useful for Claude to be able to send Slack messages and emails or make pull requests. But that's also the most damaging actions it could take. Everything on my computer is backed up, but if Claude insults my boss, that would be worse.

> So for me, it's really, really useful for Claude to be able to send Slack messages and emails or make pull requests.

Oh, I'm totally not arguing for cutting off other capabilities, I like tool use and find it to be as useful as the next person!

Just that the shell tools that will see A LOT of usage have additional guardrails added on top of them, because it's inevitable that sooner or later any given LLM will screw up and pipe the wrong thing in the wrong command - since you already hear horror stories about devs whose entire machines get wiped. Not everyone has proper backups (even though they totally should)!

Re: Go hard on agents, not on your filesystem

#247

Post by user34283 »

globular-toast wrote: post #235
user34283 wrote: post #204

Earlier quoted context omitted.

I know 'exactly' that I will need internet for research as well as installing dependencies. And I imagine it's going to be the same for most developers out there, thus the "ask for permission" model. That model seems to work quite well for millions of developers.

If you know then why do you need to be asked? A sandbox includes what you know you need in it, no more, no less.

With Codex it runs in a sandbox by default.

As we just discussed, obviously you are likely to need internet access at some point.

The agent can decide whether it believes it needs to go outside of the sandbox and trigger a prompt.

This way you could have it sandboxed most of the time, but still allow access outside of the sandbox when you know the operation requires it.

Re: Go hard on agents, not on your filesystem

#248

Post by user34283 »

imtringued wrote: post #217
user34283 wrote: post #165

Earlier quoted context omitted.

Coding agents work just fine without a sandbox. If you do use a sandbox, be prepared to endlessly click "Approve" as the tool struggles to install python packages to the right location.

I've never been annoyed by the tool asking for approval. I'm more annoyed by the fact that there is an option that gives permanent approval right next to the button I need to click over and over again. This landmine means I constantly have to be vigilant to not press the wrong button.

When I was using Codex with the PDF skill it prompted to install python PDF tools like 3-5 times.

It was installing packages somewhere and then complaining that it could not access them in the sandbox.

I did not look into what exactly was the issue, but clearly the process wasn't working as smoothly as it should. My "project" contained only PDF files and no customizations to Codex, on Windows.

Re: Go hard on agents, not on your filesystem

#250

Post by lxgr »

esperent wrote: post #244
lxgr wrote: post #225

Earlier quoted context omitted.

It works well so far, for you . Are you confident it would still work against sophisticated prompt injection attacks that override your "strongly worded message"? Strongly worded signs can be great for safety (actual mechanisms preventing undesirable actions from being taken are still much better), but are essentially meaningless for security.

I mean, that's like saying are you sure that your antivirus would prevent every possible virus? Are you sure that you haven't made some mistake in your dev box setup that would allow a hacker to compromise it? What if a thief broke i to your house and stole your laptop? That's happened to me before, much more annoying to recover from that an accidental rm rf. I do my best to keep off site back ups and don't worry abo…

> I mean, that's like saying are you sure that your antivirus would prevent every possible virus?

Yes, I'm saying it's pretty much as bad as antivirus software.

> Are you sure that you haven't made some mistake in your dev box setup that would allow a hacker to compromise it?

Different category of error: Heuristically derived deterministic protection vs. protection based on a stochastic process.

> much more annoying to recover from that an accidental rm rf.

My point is that it's a different category, not that one is on average worse than the other. You don't want your security to just stand against the median attacker.

Post reply on HN