Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't trust AI agents

nanoclaw.dev

Post reply on HN
Print view View original

61–70 of 211 posts

Re: Don't trust AI agents

#61

Post by amelius »

gyomu wrote: post #41
badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

Yeah, it’s pretty wild. Even pg is tweeting stuff like “An experienced programmer told me he's now using AI to generate a thousand lines of code an hour.“ https://x.com/paulg/status/2026739899936944495 Like if you had told pg to his face in (pre AI) office hours “I’m producing a thousand lines of code an hour”, I’m pretty sure he’d have laughed and pointed out how pointless that metric was?

Technical debt is increasing by 1,000 lines an hour.

Re: Don't trust AI agents

#62

Post by hirako2000 »

badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

More people believe a software developer job and value is in the lines of code produced.

Perhaps over half of engineering managers unconsciously or admittedly take the amount of PR and code additions as a rough but valid measure of productivity.

I recall a role in architecture, senior director asking me how come a principal engineer didn't commit any code in 2 weeks, that we pay principals a fortune.

I asked that brilliant mind whether we paid principal engineers to code or to make sure we deliver value.

Needless to say the with question went unanswered, so called Principal was fired a few months later. The entire company in fact was sold for a bargain too given it had thousands of clients globally.

The LLM can replace engineers is a phenomenon that converge from two simple facts, we haven't solved the misconception of the engineering roles. And it's the perfect scapegoat to justify layoffs.

Leaders haven't all gone insane, they answer to difficult questions with the narrative of least resistance.

Re: Don't trust AI agents

#63

Post by vitto_gioda »

Sytten wrote: post #56

I am a caveman, I don't understand the need for a personal assistant. What are you guys using it for?

I only use my own “agent” ("my", because I program it myself, since my needs are different from yours) to retrieve information about the audio I upload to it (from video calls and audio recordings). No others use cases for me

Re: Don't trust AI agents

#64

Post by medi8r »

shich wrote: post #7

the trust problem cuts both ways tho — users don't trust agents, but the bigger issue is agents trusting each other. once you have multi-agent pipelines, you're one rogue upstream output away from a cascade. sandboxing individual agents is table stakes; what's actually hard is defining trust boundaries between them

Also agents cannot trust any data whatsoever they add to their context.

This puts reading email for example as a risk.

Probably not impossible to create a worm that convinces a claw to forward it to every email address in that inbox.

And then exfiltrate all the emails.

Then do a bunch of password resets.

Then get root access to your claw.

But not just email. Github issues, wikipedia, HN etc. may be poisoned.

See https://simonw.substack.com/p/the-lethal-trifecta-for-ai-age... but there may be more trifectas than that in a claw driven future.

Re: Don't trust AI agents

#66

Post by fnord77 »

buremba wrote: post #14

My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. 2. Use incremental snapshots an…

> 1. Don't let it send emails from your personal account, only let it draft email and share the link with you.

Right now there's no way to have fine-grained draft/read only perms on most email providers or email clients. If it can read your email it can send email.

> 3. Don't let your agents see any secret. Swap the placeholder secrets at your gateway and put human in the loop for secrets you care about.

harder than you might think. openclaw found my browser cookies. (I ran it on a vm so no serious cookies found, but still)

Re: Don't trust AI agents

#67

Post by inciampati »

badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

Lines of code are nothing. It's verification that creates value.

Re: Don't trust AI agents

#69

Post by supriyo-biswas »

badsectoracula wrote: post #32

> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…

Somehow, this narrative has taken hold at multiple levels of management, especially amongst non-technical management, that "typing" was somehow the bottleneck of software engineering, reality is however more complex.

The act of "typing" code was technically mixed in with researching solutions, which means that code often took a different shape or design based on the outcome of that activity. However, this nuance has been typically ignored for faff, with the outcome that management thinks that producing X lines of code can be done "quickly", and people disagreeing with said statements are heretics who should be burned at the stake.

This is why, in my personal opinion, AI makes me only 20% productive, I often find disagreeing with the solution that it came up with and instead of having to steer it to obtain the outcome I want, I just end up rewriting the code myself. On the other hand, for prototypes where I don't care about understanding the code at all, it is more of a bigger time saver.

I could not care about the code at all, and while that is acceptable to management, not being responsible for the code but being responsible for the outcomes seems to be the same shit as being given responsibilities without autonomy, which is not something I can agree with.

Re: Don't trust AI agents

#70

Post by tao_oat »

theturtletalks wrote: post #44

Has anyone used: OpenClaw NanoClaw IronClaw PicoClaw ZeroClaw NullClaw Any insights on how they differ and which one is leading the race?

I haven't used them all but based on my partial research so far:

- OpenClaw: the big one, but extremely messy codebase and deployment

- NanoClaw: simple, main selling point is that agents spawn their own containers. Personally I don't see why that's preferable to just running the whole thing in a container for single-user purposes

- IronClaw: focused on security (tools run in a WASM sandbox, some defenses against prompt injection but idk if they're any good)

- PicoClaw: targets low-end machines/Raspberry Pis

- ZeroClaw: Claw But In Rust

- NanoBot: ~4k lines of Python, easy to understand and modify. This is the one I landed on and have been using Claude to tweak as needed for myself

Post reply on HN