Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't trust AI agents

nanoclaw.dev

Post reply on HN
Print view View original

91–100 of 211 posts

Re: Don't trust AI agents

#91

Post by arianvanp »

fnord77 wrote: post #66
buremba wrote: post #14

My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. 2. Use incremental snapshots an…

> 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. Right now there's no way to have fine-grained draft/read only perms on most email providers or email clients. If it can read your email it can send email. > 3. Don't let your agents see any secret. Swap the placeholder secrets at your gateway and put human in the loop for secrets you care about. harder than…

Literally every email client on the planet has supported `mailto:` URIs since basically the existence of the world wide web.

Just generate a mailto Uri with the body set to the draft.

Re: Don't trust AI agents

#92

Post by Doublon »

buremba wrote: post #14

My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. 2. Use incremental snapshots an…

I'd like to try a pattern where agents only have access to read-only tools. They can read you emails, read your notes, read your texts, maybe even browse the internet with only GET requests...

But any action with side-effects ends up in a Tasks list, completely isolated. The agent can't send an email, they don't have such a tool. But they can prepare a reply and put it in the tasks list. Then I proof-read and approve/send myself.

If there anything like that available for *Claws?

Re: Don't trust AI agents

#93

Post by [deleted] »

echoangle wrote: post #50

Looking at the NanoClaw GitHub README: > If you want to add Telegram support, don't create a PR that adds Telegram alongside WhatsApp. Instead, contribute a skill file (.claude/skills/add-telegram/SKILL.md) that teaches Claude Code how to transform a NanoClaw installation to use Telegram. Why would you want that? You want every user asks the AI to implement the same feature?

[deleted]

Re: Don't trust AI agents

#94

Post by daveguy »

alexhans wrote: post #29
buremba wrote: post #14

My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. 2. Use incremental snapshots an…

That's a decent practice from the lens of reducing blast radius. It becomes harder when you start thinking about unattended systems that don't have you in the loop. One problem I'm finding discussion about automation or semi-automation in this space is that there's many different use cases for many different people: a software developer deploying an agent in production vs an economist using Claude Vs a scientist thro…

Current LLMs are nowhere near qualified to be autonomous without a human in the loop. They just aren't rigorous enough. Especially the "scientist throwing a swarm to deal with common ML exploratory tasks." The judgement of most steps in the exploratory task require human feedback based on the domain of study.

> Many of the recommendations will feel too much or too little complexity for what people need and the fundamentals get lost: intent for design, control, the ability to collaborate if necessary, fast iteration due to an easy feedback loop.

Completely agreed. This is because LLMs are atrocious at judgement and guiding the sequence of exploration is critically dependent on judgement.

Re: Don't trust AI agents

#95

Post by xienze »

benatkin wrote: post #34
smallpipe wrote: post #3

Docker is not a security boundary. You’re one prompt injection away from handing over your gmail cookie.

No, but Podman is. The recent escapes at the actual container level have been pretty edge case. It's been some years since a general container escape has been found. Docker's CVE-2025-9074 was totally unnecessary and due to Docker being Docker.

The best container security in the world isn’t going to help you when the agent has credentials to third party services. Frankly, I don’t think bad actors care that much about exploiting agents to rm -rf /. It’s much more valuable to have your Google tokens or AWS credentials.

Re: Don't trust AI agents

#97

Post by [deleted] »

CuriouslyC wrote: post #87
badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

The lines of code thing isn't because we think it's a good metric, but because we have literally no good metric and we're trying to communicate a velocity difference. If you invent a new metric that doesn't have LoC's problems while being as easy to use, you'll be a household name in software engineering in short order. Also, AI is better at reading code than writing it, but the overhead to FIND code is real.

[deleted]

Re: Don't trust AI agents

#98

Post by nkzd »

medi8r wrote: post #58
nkzd wrote: post #43

As someone who only coding agents at work, can someone describe their use case for claw type agent? What do you do with it?

I want to try one to be a bit of a personal coach. Remind me to do things and check in on goals. The memory / schedule / chat thing is enough and it wont need emails or anything more dangerous.

As someone who went down so many "productivity rabbit holes" I think this is a great idea.

Re: Don't trust AI agents

#100

Post by steve1977 »

gyomu wrote: post #41
badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

Yeah, it’s pretty wild. Even pg is tweeting stuff like “An experienced programmer told me he's now using AI to generate a thousand lines of code an hour.“ https://x.com/paulg/status/2026739899936944495 Like if you had told pg to his face in (pre AI) office hours “I’m producing a thousand lines of code an hour”, I’m pretty sure he’d have laughed and pointed out how pointless that metric was?

We all know that a thousand parentheses would be better metric.
Post reply on HN