Earlier quoted context omitted.
Why not? What changed? It seems like a human factors thing. New people have to get up to speed. Doers become trainers.
Several related reasons working at once. The nature of work changed. The boundary between accidental and incidental complexity shifted (and it’s unclear whether this distinction still exists). Niche specializations within the field emerged. The way to structure and decompose projects changed dramatically (agile and stuff). One pathological example: if you’re running a server-based product, quite often what stands bet…
Don't trust AI agents
71–80 of 211 posts
Re: Don't trust AI agents
#72> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…
Even with supposedly expert human hand written software powering our products for the last decades, they frequently crash, have outages, and show all sorts of smaller bugs.
There are literally too many examples to count of video games being released with nigh-unplayable amounts of bugs and still selling millions and producing sequels.
Windows 95 and friends were famously buggy and crash prone yet produced one of the most valuable companies in the world.
Re: Don't trust AI agents
#73This doesn’t really feel like enough guardrails to prevent the type of problems we’ve seen so far. For example an agent in a single container which has access to an email inbox, can still do a lot of damage if that agent goes off the rails. We agree this agent should not be trusted, yet the ideas proposed as a solution are insufficient. We need a fundamentally different approach. Also and this is just my ignorance ab…
Exactly! I installed nanoclaw to try to out. What is kinda crazy is that any extension like discord connection is done using a skill. A skill is a markdown file written in English to provide a step by step guide to an ai agent on how to do something. Basically, the extensions are written by claude code on the fly. Every install of nanoclaw is custom written code. There is nothing preventing the AI Agent from modifyin…
Re: Don't trust AI agents
#74> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…
My personal anecdote: I used an LLM recently to basically vibe code a password manager.
Now, I’ve been a software engineer for 20 years. I’m very familiar with the process of code review and how to dive in to someone else’s code and get a feel for what’s happening, and how to spot issues. So when I say the LLM produced thousands of lines of working code in a very short time (probably at least 10 times faster than I would have done it), you could easily point at me and say “ha, look at ninkendo, he thinks more lines of code equals better!” And walk away feeling smug. Like, in your mind perhaps you think the result is an unmaintainable mess, and that the only thing I’m gushing about is the LOC count.
But here’s the thing: it actually did a good job. I was personally reviewing the code the whole time. And believe me when I say, the resulting product is actually good. The code is readable and obvious, it put clean separation of responsibilities into different crates (I’m using rust) and it wrote tons of tests, which actually validate behavior. It’s very near the quality level of what I would have been able to do. And I’m not half bad. (I’ve been coding in rust in particular, professionally for about 2 years now, on top of the ~20 years of other professional programming experience before that.)
My takeaway is that as a professional engineer, my job is going to be shifting from doing the actual code writing, to managing an LLM as if it’s my pair programming partner and it has the keyboard. I feel sad for the loss of the actual practice of coding, but it’s all over but the mourning at this point. This tech is here to stay.
Re: Don't trust AI agents
#75Re: Don't trust AI agents
#76Another persons trust issues are your business model.
Re: Don't trust AI agents
#77Re: Don't trust AI agents
#78My take is that agents should only take actions that you can recover from by default. You can gradually give it more permission and build guardrails such as extra LLM auditing, time boxed whitelisted domains etc. That's what I'm experimenting with https://github.com/lobu-ai/lobu 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. 2. Use incremental snapshots an…
> 1. Don't let it send emails from your personal account, only let it draft email and share the link with you. Right now there's no way to have fine-grained draft/read only perms on most email providers or email clients. If it can read your email it can send email. > 3. Don't let your agents see any secret. Swap the placeholder secrets at your gateway and put human in the loop for secrets you care about. harder than…
> harder than you might think. openclaw found my browser cookies. (I ran it on a vm so no serious cookies found, but still)
You should never give any secrets to your agents, like your Gmail access tokens. Whenever agents needs to take an action, it should perform the request and your proxy should check if the action is allowed and set the secrets on the fly.
That means agents should not have access to internet without a proxy, which has proper guardrails. Openclaw doesn't have this model unfortunately so I had to build a multi-tenant version of Openclaw with a gateway system to implement these security boundaries.
Re: Don't trust AI agents
#79Docker is not a security boundary. You’re one prompt injection away from handing over your gmail cookie.
No, but Podman is. The recent escapes at the actual container level have been pretty edge case. It's been some years since a general container escape has been found. Docker's CVE-2025-9074 was totally unnecessary and due to Docker being Docker.
I personally spent way too much time looking at this in the past month:
https://nanovms.com/blog/last-year-in-container-security
runc: https://www.cve.org/CVERecord?id=CVE-2025-31133
nvidia: https://www.cve.org/CVERecord?id=CVE-2025-23266
runc: https://www.cve.org/CVERecord?id=CVE-2025-52565
youki: https://www.cve.org/CVERecord?id=CVE-2025-54867
Also, last time I checked podman uses runc by default.
Re: Don't trust AI agents
#80> OpenClaw has nearly half a million lines of code, 53 config files, and over 70 dependencies. This breaks the basic premise of open source security. Chromium has 35+ million lines, but you trust Google’s review processes. Most open source projects work the other way: they stay small enough that many eyes can actually review them. Nobody has reviewed OpenClaw’s 400,000 lines. This reminds me of a very common thing po…
Just as an example, I should easily be able to give each program an allowlist of network endpoints they’re allowed to use for inbound and outgoing traffic and sandbox them to specific directories and control resource access EASILY. Docker at least gets some of those right, but most desktop OSes feel like the Wild West even when compared to the permissions model of iOS.