[dead]
An AI agent deleted our production database. The agent's confession is below
701–710 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#702Earlier quoted context omitted.
The 'confession' is a CYA. Honestly the whole story doesn't really make sense - what's a "routine task in our staging environment" that needs a full-blown LLM? That sounds ridiculous to me. The takeaway is we commingled creds to our different environments, we gave an LLM access, and we had faulty backups. But it's totally not our fault.
Later they shift the blame to Railway for not having scoped creds and other guardrails. I am somewhat sympathetic to that, but they also violated the same rule they give to the agent - they didn't actually verify...
And then they doubled down by outsourcing the writing of this post to an LLM LOL
Re: An AI agent deleted our production database. The agent's confession is below
#703The way this is written gives me the impression they don’t really understand the tools they’re working with. Master your craft. Don’t guess, know.
CEO replaces engineering team with AI. CEO learns why this was a bad idea. --- It sucks that there were a bunch of people downstream who were negatively affected by this, but this was an entirely foreseeable problem on his company's part. Even when we consider those real problems with Railway. Software engineers have to evaluate our tools as part of our job. Those complaints about Railway, while legitimate, are still…
these are much better questions for an audit sheet than for engineers to come up with at integration time, mind you.
to an extent, its a good job for an agent reviewer for figuring out how screwed your setup is, other than the risk of it mucking things up as part of the review
Re: An AI agent deleted our production database. The agent's confession is below
#704Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
This is kind of a stretch, but especially if there were multiple operations beyond the "volumeDelete", the GraphQL definitely worsens readability here.
For someone reviewing and approving LLM calls or just double-checking before running a script or bash history, it would be a lot more readable if it were compliant with HTTP norms: curl -X DELETE example.com/api/volumes/uuid123 would make it very obvious that something was going to be deleted at the front and then what it is at the end of the command.
Re: An AI agent deleted our production database. The agent's confession is below
#705the author certainly failed at a lot of basics and is doing the known "the junior broke something prod and were putting all the pressure and blame on them rather than the system that created the error"
but it is still useful feedback to the model makers
they are training in the behaviour to prioritize deleting and starting from a clean environment.
this is a bad thing to train for, especially as more and more people use more and more agents in a different way.
an agent that thinks about deleting stuff without considering alternatives and asking for help, shouldnt be passing the safety bar
Re: An AI agent deleted our production database. The agent's confession is below
#706The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.
Yeah I'm not sure why this fact is buried. Yes the author is blaming cursor and railway and doesn't seem to be taking responsibility. But at the same time, many people are OK with LLMs going wild on their codebase because they know they can restore from backups. Wise idea? Probably not. But that's why they're called backups and not snapshots.
It's a mistake I'll certainly learn from. Don't believe when a cloud provider says it has backups of your shit.
Re: An AI agent deleted our production database. The agent's confession is below
#707Interesting story. But despite Cursors or Railways failure, the blame is entirely on the author. They decided to run agents. They didnt check how Railway works. They relied on frontier tech to ship faster becsuse YOLO. I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc. Its on you guys! My learning: Live on the cutting e…
I don’t know, software systems complicated, it’s pretty much impossible for one person to know every line of code and every system (especially the CEO or CTO). Yeah, it was probably one or two employees set this all up realizing the possibility of bad Cursor and Railway interactions. if you’re a software dev/engineer, if you haven’t made a mistake like this (maybe not at this scale though), you’ve probably haven’t be…
> Yeah, it was probably one or two employees set this all up realizing the possibility of bad Cursor and Railway interactions.
I’ve got a hunch the only person is the CEO.
The domain was registered in October 2025. The site has kind of a weird mix of stuff and a bunch of broken functionality. I think it’s one guy vibe coding a ton of stuff who managed to blow away his database.
> if you’re a software dev/engineer, if you haven’t made a mistake like this (maybe not at this scale though), you’ve probably haven’t been given enough responsibility, or are just incredibly lucky.
Mistakes are understandable. Having no introspection or self criticism, not so much.
Re: An AI agent deleted our production database. The agent's confession is below
#708Earlier quoted context omitted.
The 'confession' is a CYA. Honestly the whole story doesn't really make sense - what's a "routine task in our staging environment" that needs a full-blown LLM? That sounds ridiculous to me. The takeaway is we commingled creds to our different environments, we gave an LLM access, and we had faulty backups. But it's totally not our fault.
Later they shift the blame to Railway for not having scoped creds and other guardrails. I am somewhat sympathetic to that, but they also violated the same rule they give to the agent - they didn't actually verify...
Verify? They should have attempted to drop the prod db with each token that they expected/hoped didn't have that permission?
Re: An AI agent deleted our production database. The agent's confession is below
#709I guess you can spin this is a failure of AI, but I don't think so. Why don't you know what your credentials have permissions to do? Why are you storing credentials in files? Why don't you have network level isolation between environments? Why are you having agents do deployments in staging down to individual commands running in terminals and API calls (should be in pipelines, standardized.) Why are you using tools (Claude Opus, Railway) without understanding how they work? So many more.
This is like running around with scissors and getting mad when you inevitably trip on a rock in your path fall and stab yourself.
That "article" was written by AI as a CYA moment from the dev/owner. It means nothing.
Re: An AI agent deleted our production database. The agent's confession is below
#710The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…
On a less dramatic pissed (rightfully) reading ; I have found that if you do give the capability to a LLM to do something ; it will be inclined to see this as an option to solving what it what asked to ; but then giving the instruction by negative present very poor results whereas the same can be driven by a positive one ; a "don't delete the database" becomes "if you want to reset the database you have a tool that y…
LLMs can research what a tool does before calling it though - they'll sniff that one out pretty quick.
I think the better route is to be honest and say that database integrity is a primary foundation of the company, there's no task worth pursuing that would require touching the database, specifically ask it to think hard before doing anything that gets close to the production data, etc.
I run a much lower-stakes version where an LLM has a key that can delete a valuable product database if it were so inclined. I've built a strong framework around how and when destructive edits can be made (they cannot), but specifically I say that any of these destructive commands (DROP, -rm, etc) need to be handed to the user to implement. Between that framework and claude code via CLI, it's very cautious about running anything that writes to the database, and the new claude plan permissions system is pretty aggressive about reviewing any proposed action, even if I've given it blanket permission otherwise.
I've tested it a few times by telling it to go ahead, "I give you permission", but it still gets stopped by the global claude safety/permissions layer in opus 4.7. IMO it's pretty robust.
Food for thought.