Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

821–830 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#821
This is why I still have a "human gate" rule: any destructive operation needs a second pair of eyes, even if the first pair is an AI.

The scariest part isn't that an AI deleted a db — it's that the infra allowed it. No backup? No IAM restrictions? No staging environment that mirrors prod but can't touch it?

AI agents are force multipliers. That includes force multiplying your mistakes.

Re: An AI agent deleted our production database. The agent's confession is below

#822
I definitely empathize but:

> There is no role-based access control for the Railway API — every token is effectively root. The Railway community has been asking for scoped tokens for years. It hasn't shipped.

Why the hell did you go with their stack then? RBAC should be table stakes for such a solution, no?

Re: An AI agent deleted our production database. The agent's confession is below

#823
The real hard question is: "SO WHAT?". Is anybody going to stop using agents? No, it takes you out of competition fast. Is anybody can do anything about _how_ they use agents or _how_ they design guardrail better? No, because nobody knows how. Is it gonna make agents' authors fix it? No, because they are also invested hugely into this research and so far they don't know how to solve it either.

Re: An AI agent deleted our production database. The agent's confession is below

#824
Agent permissions layer are broken. We need better a permissions layer that doesn’t get in the way but stops destructive commands. Devs get pushed into running yolo mode cause classifying allow / deny by command is not enough. A sandbox would not have prevented this either.

“nah” is a context aware permission layer that clasifies commands based on what they actually do

nah exposes a type taxonomy: filesystem_delete, network_write, db_write, etc

so commands gets classified contextually:

git push ; Sure. git push --force ; nah?

rm -rf __pycache__ ; Ok, cleaning up. rm ~/.bashrc ; nah.

curl harmless url ; sure. curl destroy_db ; nah.

https://github.com/manuelschipper/nah

Better permissions layers is part of the answer here, and a space that has been only narrowly explored.

Re: An AI agent deleted our production database. The agent's confession is below

#827
Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.

Re: An AI agent deleted our production database. The agent's confession is below

#828

I definitely empathize but: > There is no role-based access control for the Railway API — every token is effectively root. The Railway community has been asking for scoped tokens for years. It hasn't shipped. Why the hell did you go with their stack then? RBAC should be table stakes for such a solution, no?

Ironic given that real railways invented the access control "token" for safety purposes in the middle of the nineteenth century: https://en.wikipedia.org/wiki/Token_(railway_signalling)

Re: An AI agent deleted our production database. The agent's confession is below

#829

Earlier quoted context omitted.

> They predict text not obey our orders. Those are the same thing in this case. The latter is just an extremely reductionist description of the mechanics behind the former.

They are not in fact the same thing, and the difference is important. They are certainly marketed as if they think, learn and follow orders, but they do not.

The result of "predicting text" is that they obey orders, just like the result of "random electrochemical impulses in synapses" is that you typed your comment.

You can always reduce high-level phenomena to lower-level mechanisms. That doesn't mean that the high-level phenomenon doesn't exist. LLMs are obviously able to understand and follow instructions.

Re: An AI agent deleted our production database. The agent's confession is below

#830
post #827

Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.

Shouting at them is like shouting at your chainsaw after it just chopped off your foot
Post reply on HN