Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

671–680 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#671

Post by jiggawatts »

herdymerzbow wrote: post #659
9dev wrote: post #416

Earlier quoted context omitted.

It’s awful. "We had no clue this token had the permission to delete stuff!" - well buddy you issued it without deciding on permissions, it’s your job to assert that. Your latest recoverable backup is three months old? The rule is 3-2-1, you didn’t follow it. Nobody else to blame but yourself. And on and on he rambles…

This is what stood out to me. I've no actual experience operating in this area, but I have been a very grateful user recipient of backups. Anyway, I thought backups were a nightly thing....? Particularly if that data is essentially your business. Presumably it costs a bit to set up but it surely it's unacceptable not to set it up?

Hourly or even more frequently is commonplace because transaction log backups are relatively cheap to take and keep, especially in the era of blob storage. In the olden days, tape drives couldn't keep up this level of backup schedule because they're bad at frequent stop-starts and interleaving a bunch of unrelated transaction logs would make recovery very slow. This just isn't an issue any more and anybody competent is backing up multiple times per day.

Re: An AI agent deleted our production database. The agent's confession is below

#672

Post by oxag3n »

pigbearpig wrote: post #534
oxag3n wrote: post #509

Why so many comments blame the author? If AI is just a tool, just like a database console, would you blame user for entire database loss if he just tried to update a single row in a table?

It's situational. The blame on how the tool was used and whether this was negligence. If I hit someone with my car because I was looking at my phone, it's not the tools fault. If I hit someone because my brakes failed due to a manufacturing defect, sure blame the tool. In this situation, the author didn't understand the API key they created. They also likely told the AI it could do a bunch of things (I have claude co…

I did read it differently:

> The agent ran this command: ...

> No confirmation step. No "type DELETE to confirm."...

I thought the author expected the Agent to ask for confirmation before running this command.

Re: An AI agent deleted our production database. The agent's confession is below

#673

Post by devmor »

Ridiculous.

An AI agent didn’t delete your database - poor security policy did. An AI agent might have been the factor this time, but it could have just as easily been a malicious supply chain dependency or an angry employee.

You know what the very first thing I did when I started using agentic LLMs was? Isolate their surface area. Started with running them in a docker container with mounted directories. Now I have a full set of tools for agent access - but that was just to protect my hobby projects.

Re: An AI agent deleted our production database. The agent's confession is below

#674

Post by jiggawatts »

crazygringo wrote: post #313
hu3 wrote: post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

Yup, this is bizarre. A top use case for needing a backup is when you accidentally delete the original. You need to be able to delete backups too, of course, but that absolutely needs to be a separate API call. There should never be any single API call that deletes both a volume and its backups simultaneously. Backups should be a first line of defense against user error as well. And I checked the docs -- they're call…

Azure SQL Database did this too for a while until enough companies complained about losing their data and their backups with a single action.

Re: An AI agent deleted our production database. The agent's confession is below

#675

Post by frm88 »

fathermarz wrote: post #470
827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

Completely agree. This is a harness problem, not a model problem. The model is rarely the issue these days

I don't know. To me, this is a human problem. Not only has the model access to the production database, they have the backups online on the same volume, have an offline backup 3 month old. This is an accumulation of bad practices, all of them human design failures. Instead of sitting down and rethinking their entire backup strategy they go public on twitter and blame a probabilistic machine doing what is within its parameters to do. I bet, even that failure could have been avoided, were more care given to what they do.

Re: An AI agent deleted our production database. The agent's confession is below

#676

Post by ReptileMan »

There are similarities between this and the Titan submersible guy - real men test in production.

If an agent has a production data access or token - that is deep failure in your workflow. If you don't have offsite backup - deep failure in your workflow.

Re: An AI agent deleted our production database. The agent's confession is below

#677

Post by subhobroto »

M_bara wrote: post #328

That is why i insist on 1. Streaming replication whether from RDS or my own DB 2. Db dumps shipped to s3 using write only creds or something like rsync. Streaming gets you PIT recovery while DB dumps give me daily snapshots stored daily for 14 days. An aside: 15 or so years ago, a work colleague made a mistake and dropped the entire business critical DB - at a critical internet related company - think of continent wi…

> Streaming replication whether from RDS

Are you using AWS RDS Custom to receive the WAL Streams or are you using something like Pigsty? Really curious about the actual specifics

Re: An AI agent deleted our production database. The agent's confession is below

#679

Post by shruubi »

Honestly, things like this just depress me. Someone makes a mistake and then they try to cover themselves by saying "Yeah I am somewhat to blame, but look at all these other things that are more to blame". They seem responsible by appearing to take accountability but in actuality are pushing accountability onto everyone else before themselves.

Then, to get clicks and attention we then ask the AI to write some kind of "confession". It's a probability engine, it has no thoughts or feelings you can hurt or shame into doing better, it has no long term memory to burn the embarrassment of this into and in fact given the same circumstances it is probable that the agent would do the same thing again and again no matter how many confessions you have it write or how mean you write to it.

Ultimately, you are the operator of the machine and the AI, and despite what OpenAI/Anthropic/Whomever say, you are required to exist because the machine cannot operate without you being there nor can it be accountable for what it does.

Re: An AI agent deleted our production database. The agent's confession is below

#680

Post by nonfamous »

smrtinsert wrote: post #503
827a wrote: post #261

The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…

> "NEVER FUCKING GUESS" It's very hard to treat this post seriously. I can't imagine what harness if any they attempted to place on the agent beyond some vibes. This is "most fast and absolutely destroy things" level thinking. That the poster asks for journalists to reach out makes it like a no news is bad news publicity grab. Just gross. The AI era is turning about to be most disappointing era for software engineeri…

I'd be interested to learn where those words exist in Cursor's context. My assumption was that it was part of the Cursor agent harness, but it's just as likely it was in the user instructions.
Post reply on HN