Live data from Hacker News

Don't trust AI agents

nanoclaw.dev

11–20 of 211 posts

Re: Don't trust AI agents

#11

My assistant has no permissions at all and is just as useful. All it needs is todo, reminders and websearch (and maybe a browser but ymmv).

> websearch (and maybe a browser

Your assistant can literally be told what to do and how to hide it from you. I know security is not a word in slopware but as a high-level refresher - the web is where the threats are.

Re: Don't trust AI agents

#12
post #9

My assistant has no permissions at all and is just as useful. All it needs is todo, reminders and websearch (and maybe a browser but ymmv).

> no permissions at all > and maybe a browser does not compute

I suspect OP actually means 'cannot access anything locally' by 'no permissions'.

Re: Don't trust AI agents

#13

This doesn’t really feel like enough guardrails to prevent the type of problems we’ve seen so far. For example an agent in a single container which has access to an email inbox, can still do a lot of damage if that agent goes off the rails. We agree this agent should not be trusted, yet the ideas proposed as a solution are insufficient. We need a fundamentally different approach. Also and this is just my ignorance ab…

Wouldn't you get >50% of the usefulness and 0% of the risk if you add read+draft permissions for the email connection through a proxy or oauth permissions? Then your claw can draft replies and you have to manually review+send. It's not a perfect PA that way, but could still be better than doing everything yourself for the vast majority of people who don't have a PA anyway?

It feels like, just like SWEs do with AI, we should treat the claw as an enthusiastic junior: let it do stuff, but always review before you merge (or in this case: send).

Re: Don't trust AI agents

#14
My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu

1. Don't let it send emails from your personal account, only let it draft email and share the link with you.

2. Use incremental snapshots and if agent bricks itself (often does with Openclaw if you give it access to change config) just do /revert to last snapshot. I use VolumeSnapshot for lobu.ai.

3. Don't let your agents see any secret. Swap the placeholder secrets at your gateway and put human in the loop for secrets you care about.

4. Don't let your agents have outbound network directly. It should only talk to your proxy which has strict whitelisted domains. There will be cases the agent needs to talk to different domains and I use time-box limits. (Only allow certain domains for current session 5 minutes and at the end of the session look up all the URLs it accessed.) You can also use tool hooks to audit the calls with LLM to make sure that's not triggered via a prompt injection attack.

Last but last least, use proper VMs like Kata Containers and Firecrackers. Not just Docker containers in production.

Re: Don't trust AI agents

#15
post #10

Why does OpenClaw have 800,000+ lines of code?? Isn't it just a connector for LLM APIs and other tools?

They are probably counting dependencies. Also, it's vibe coded, what do you expect!

I used to think that LLMs would replace humans but now I'm confident that I'll have a job in the future cleaning up slop. Lucky us.

Re: Don't trust AI agents

#16
post #10

Why does OpenClaw have 800,000+ lines of code?? Isn't it just a connector for LLM APIs and other tools?

They are probably counting dependencies. Also, it's vibe coded, what do you expect! I used to think that LLMs would replace humans but now I'm confident that I'll have a job in the future cleaning up slop. Lucky us.

I did a cloc check on it and it does seem to have 800k lines of typescript. So unless they are vendoring dependencies it's actually as insane as it sounds.

Re: Don't trust AI agents

#17
post #13

This doesn’t really feel like enough guardrails to prevent the type of problems we’ve seen so far. For example an agent in a single container which has access to an email inbox, can still do a lot of damage if that agent goes off the rails. We agree this agent should not be trusted, yet the ideas proposed as a solution are insufficient. We need a fundamentally different approach. Also and this is just my ignorance ab…

Wouldn't you get >50% of the usefulness and 0% of the risk if you add read+draft permissions for the email connection through a proxy or oauth permissions? Then your claw can draft replies and you have to manually review+send. It's not a perfect PA that way, but could still be better than doing everything yourself for the vast majority of people who don't have a PA anyway? It feels like, just like SWEs do with AI, we…

Agent can still "forgot password" on many accounts. Or magic link.

Re: Don't trust AI agents

#19

Earlier quoted context omitted.

They are probably counting dependencies. Also, it's vibe coded, what do you expect! I used to think that LLMs would replace humans but now I'm confident that I'll have a job in the future cleaning up slop. Lucky us.

I did a cloc check on it and it does seem to have 800k lines of typescript. So unless they are vendoring dependencies it's actually as insane as it sounds.

Christ their repo is an absolute nightmare. There's new issues and PRs being posted practically every minute, and I assume 99% of them are from agents given the target demographic. Just full-auto vibeslop from all barrels 24/7.

Even if we count the repos whole lifetime, including when it wasn't so active, the averages are still absurd.

96 days / (4,239+9,170) issues = one issue every 10 minutes

96 days / (5,082+10,221) pull requests = one PR every 9 minutes

Re: Don't trust AI agents

#20
> The container boundary is the hard security layer — the agent can’t escape it regardless of configuration

I thought containers were never a proper hard security barrier? It’s barrier so better than not having it, if course.

Post reply on HN