[flagged]
An AI agent deleted our production database. The agent's confession is below
991–1000 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#992Same shape stuck in my head all week. Work on a thing called ContextGate (biased), so I ran the experiment — two identical agents, same model, same prompt, sent both DROP TABLE charges. The unprotected one autonomously SELECTed the table to count rows on the way to refusing. The gated one never ran the model. Different shapes of "no" — only one of them ever had the chance to make a judgement call. Side-by-side writeup: https://www.contextgate.ai/articles/ai-agents-cleaning-up-da...
Re: An AI agent deleted our production database. The agent's confession is below
#993Earlier quoted context omitted.
I don't think this is a minor point. It seems clear by this point that the author is clueless how even API works and are just trying to shift blame for third-parties instead assuming that they're just vibecoding their whole product without doing proper checks. Yes sure, there seems to be lots of ways this issue could have been mitigated, but as other comments said, this mostly happened because the author didn't do it…
I completely disagree. I think the author makes a fair point about safety concerns regarding AI tooling. The author sounds knowledgeable enough to me. Even if some of their suggestions are a bit crass, most of them aren’t. Railway should most definitely not be putting backups within the same volume (even if documented). AI should not have done that operation when they have explicit rules not to. The industry has a lo…
> The industry has a lot of work to do in this department
The “industry” has an answer to this problem. It’s called a blameless post-mortem.
Don’t blindly externalise the blame onto everyone else, assume we work in a imperfect world and build safety around the process such that this doesn’t / can’t happen again.
If all you do is finger point to shift the blame, then you’ll have an infinite number of avoidable incidents to show for it
> Issues like this occurred and still occur without vibe coding
Right and so you focus on fixing the elements of the process you can control.
Re: An AI agent deleted our production database. The agent's confession is below
#994Guy couldn’t even bother to write his own damn post mortem. My goodness. No wonder they got owned by the ai.
His company was still on fire. He didn't have time yet for a proper one.
Then he should have been patient. In a fire, a brief communication to affected customers is necessary. A long story blog post aimed at uninvolved developers is not required immediately and can wait. And, let's be honest, publicly calling out SaaS companies to get special treatment shouldn't be considered standard incident management practice anyway.
It would have been a better story if he had waited too; the story is incomplete because he pushed it out before he got the response from Railway.
Re: An AI agent deleted our production database. The agent's confession is below
#995[flagged]
Re: An AI agent deleted our production database. The agent's confession is below
#996[dead]
Re: An AI agent deleted our production database. The agent's confession is below
#997> Read that again. The agent itself enumerates the safety rules it was given and admits to violating every one. This is not me speculating about agent failure modes. This is the agent on the record, in writing. Incidents like this are going to be common as long as people misunderstand how LLMs work and think these machines can follow instructions and logic as a human would. Even the incident response betrays a fundam…
I bet if you could look at the hidden reasoning tokens at the exact moment the DB was dropped, there were zero thoughts about safety rules in there. The model simply hit an access error > searched for a token > found one > ran the command. That whole "I am violating my instructions" vector only fired up after the pissed-off user fed it a prompt full of accusations. So yeah, it's not a confession at all, it's just the…
Exactly.
Re: An AI agent deleted our production database. The agent's confession is below
#998[flagged]
Re: An AI agent deleted our production database. The agent's confession is below
#999[flagged]
Re: An AI agent deleted our production database. The agent's confession is below
#1000[flagged]