Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

311–320 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#311

Post by jumpconc »

exe34 wrote: post #173
hu3 wrote: post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

If your backup is inside the same thing you backed up, you don't have a backup. You have an out of date copy.

All my backups are inside the same universe as what is being backed up. A boundary must be drawn somewhere and this is one of many reasonable boundaries. As I understand it, the backup isn't "inside" the volume but is attached to it so that deleting the volume deletes the backups.

Re: An AI agent deleted our production database. The agent's confession is below

#312

Post by Negitivefrags »

chrsw wrote: post #264
yongjik wrote: post #198

Earlier quoted context omitted.

> It is fundamental to language modeling that every sequence of tokens is possible. This is just trivially wrong that I don't understand why people repeat it. There are many valid criticisms of LLM (especially the LLMs we currently have), this isn't one of them. It's akin to saying that every molecules behave randomly according to statistical physics, so you should expect your ceiling to spontaneously disintegrate an…

Ceilings do fall on people. LLMs do delete production databases. Will these things always inevitably happen? No, but the moment it does happen to someone I doubt they will be thinking about probabilities or Murphy's law or whatever. I guess the question is, since we know these things can happen, however unlikely, what mitigations should be in place that are commensurate with the harms that might result?

> I guess the question is, since we know these things can happen, however unlikely, what mitigations should be in place that are commensurate with the harms that might result?

This isn't a defence of using LLMs like this, but this statement taken at face value is a source of a lot of terrible things in the world.

This is the kind of stuff that leads to a world where kids are no longer able to play outside.

Re: An AI agent deleted our production database. The agent's confession is below

#313

Post by crazygringo »

hu3 wrote: post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

Yup, this is bizarre. A top use case for needing a backup is when you accidentally delete the original.

You need to be able to delete backups too, of course, but that absolutely needs to be a separate API call. There should never be any single API call that deletes both a volume and its backups simultaneously. Backups should be a first line of defense against user error as well.

And I checked the docs -- they're called backups and can be set to run at a regular interval [1]. They're not one-off "snapshots" or anything.

[1] https://docs.railway.com/volumes/backups

Re: An AI agent deleted our production database. The agent's confession is below

#314

Post by AndrewDucker »

keeda wrote: post #290
xmodem wrote: post #273

Earlier quoted context omitted.

Don't anthropomorphize the language model. If you stick your hand in there, it'll chop it off. It doesn't care about your feelings. It can't care about your feelings.

Actually I think the opposite advice is true. Do anthropomorphize the language model, because it can do anything a human -- say an eager intern or a disgruntled employee -- could do. That will help you put the appropriate safeguards in place.

No, because the safeguards should be appropriate to an LLM, not to a human.

(The LLM might act like one of the humans above, but it will have other problematic behaviours too)

Re: An AI agent deleted our production database. The agent's confession is below

#316

Post by rglullis »

keeda wrote: post #290
xmodem wrote: post #273

Earlier quoted context omitted.

Don't anthropomorphize the language model. If you stick your hand in there, it'll chop it off. It doesn't care about your feelings. It can't care about your feelings.

Actually I think the opposite advice is true. Do anthropomorphize the language model, because it can do anything a human -- say an eager intern or a disgruntled employee -- could do. That will help you put the appropriate safeguards in place.

An eager intern can not be working for hundreds of millions of customers at the same time. An LLM can.

A disgruntled employee will face consequences for their actions. No one at Anthropic, OpenAI, xAI, Google or Meta will be fired because their model deleted a production database from your company.

Re: An AI agent deleted our production database. The agent's confession is below

#318

Post by blurbleblurble »

hu3 wrote: post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

"The author's confession is above..."

Re: An AI agent deleted our production database. The agent's confession is below

#319

Post by crazygringo »

As unfortunate as this outcome was, the docs clearly state that you should have a recovery period of 48 hours (strange the post doesn't mention it):

> Deletion and Restoration

> When a volume is deleted, it is queued for deletion and will be permanently deleted within 48 hours. You can restore the volume during this period using the restoration link sent via email.

> After 48 hours, deletion becomes permanent and the volume cannot be restored.

https://docs.railway.com/volumes/reference

Re: An AI agent deleted our production database. The agent's confession is below

#320

Post by hun3 »

jumpconc wrote: post #303
amluto wrote: post #214

Earlier quoted context omitted.

Humans can do one thing that AI agents are 100% completely incapable of doing: being accountable for their actions.

You haven't met certain humans. Not all humans have internal capacity for accountability. The real meaning of accountability is that you can fire one if you don't like how they work. Good news! You can fire an AI too.

But it's still a bit more difficult to sue them for leaking your company's data.

At least for now.

Post reply on HN