Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

381–390 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#381

Post by annoyingcyclist »

lmf4lol wrote: post #18

Interesting story. But despite Cursors or Railways failure, the blame is entirely on the author. They decided to run agents. They didnt check how Railway works. They relied on frontier tech to ship faster becsuse YOLO. I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc. Its on you guys! My learning: Live on the cutting e…

I kept reading and reading to find the part where the author took responsibility for any part of this, then I got to the end.

Re: An AI agent deleted our production database. The agent's confession is below

#382

Post by theshrike79 »

nubinetwork wrote: post #245
jeremyccrane wrote: post #220

Earlier quoted context omitted.

This is a huge issue.

A lot of VPSes operate this way as well, delete the VM, lose your backups.

A "backup" like that should be called a "snapshot".

Re: An AI agent deleted our production database. The agent's confession is below

#383

Post by theshrike79 »

jumpconc wrote: post #311
exe34 wrote: post #173

Earlier quoted context omitted.

If your backup is inside the same thing you backed up, you don't have a backup. You have an out of date copy.

All my backups are inside the same universe as what is being backed up. A boundary must be drawn somewhere and this is one of many reasonable boundaries. As I understand it, the backup isn't "inside" the volume but is attached to it so that deleting the volume deletes the backups.

Can we at least agree to draw the line so that if a single call can delete the live data AND all backups, they shouldn't be called "backups", but rather snapshots?

Re: An AI agent deleted our production database. The agent's confession is below

#385

Post by hn_throwaway_99 »

gigatree wrote: post #333
827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

He’s not necessarily anthropomorphizing it, he’s showing that it went against every instruction he gave it. Sure concepts like “confession” technically require a conscious mind, but I think at this point we all know what someone means when they use them to describe LLM behavior (see also “think”, “say”, “lie” etc)

The entire post looks like an exercise in CYA. To be fair, I have a ton of sympathy for the author, but I think his response totally misses the point. In my mind he is anthropomorphizing the agent in the sense of "I treated you like a human coworker, and if you were a human coworker I'd be pissed as hell at you for not following instructions and for doing something so destructive."

I would feel a lot differently if instead he posted a list of lessons learned and root cause analyses, not just "look at all these other companies who failed us."

Re: An AI agent deleted our production database. The agent's confession is below

#386

Post by altmanaltman »

keeda wrote: post #290
xmodem wrote: post #273

Earlier quoted context omitted.

Don't anthropomorphize the language model. If you stick your hand in there, it'll chop it off. It doesn't care about your feelings. It can't care about your feelings.

Actually I think the opposite advice is true. Do anthropomorphize the language model, because it can do anything a human -- say an eager intern or a disgruntled employee -- could do. That will help you put the appropriate safeguards in place.

it cannot go to the washroom and cry while pooping. And thats just one of the things that any human can do and AI cannot. So no it cannot do anything a human can do, the shared exmaple being one of them.

And thats why we dont have AI washrooms because they are not alive or employees or have the need to excrete.

Re: An AI agent deleted our production database. The agent's confession is below

#388

Post by uberduper »

I previously worked at a managed database as a service company. On more than one occasion during my time there, a junior engineer deleted a customers database and at least one time one of our most senior dbas made it unrecoverable. Never got such straight forward confessions out of them.

Re: An AI agent deleted our production database. The agent's confession is below

#389

Post by andy81 »

easton wrote: post #79
ad_hockey wrote: post #28

Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…

AWS actually has a thingy on some services called “deletion protection” to prevent automation from accidentally wiping resources the user didn’t want it to (you set the bit, and then you need to make a separate api request to flip the bit back before continuing). I think it’s designed for things like Terraform or CloudFormation where you might not realize the state machine decided your database needed to be replaced…

Agents will happily automate away intentional friction like a confirm prompt, even if you organise it as multiple API calls.

The fix needs to be permissions rather than ergonomics.

Re: An AI agent deleted our production database. The agent's confession is below

#390

Post by rednb »

As someone who uses quite a couple of different AI providers (codex, glm, deepseek, claude premium among others), i've noticed that claude tends to move too fast and execute commands without asking for permission.

For example, if i ask a question regarding an implementation decision while it is implementing a plan, it answers (or not) and immediately proceeds to make changes it assumes i want. Other models switch to chat mode, or ask for the best course of action.

Once this is said, i am not blaming Anthropic For that one, because IMHO the OP has taken a lot of risks and failed to design a proper backup and recovery strategy. I wish them to recover from this though, this must be a very stressful situation for them.

Post reply on HN