Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

151–160 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#151
post #98

Man, such a difference between a human whoops and an AI. Had a junior dev hork all environments, when the script they thought worked in nonprod... did not modify an index like they expected, they were quickly able to wipe out everything else in every environment and every data center. It was such a teachable moment. She was my very first hire when I was asked to build a team. Crazy careful with trust, but verify on t…

And it’s not the junior’s fault when they do it either.

Have some controls in place. Don’t rely on nobody being dumb enough to do X. And that includes LLMs.

Re: An AI agent deleted our production database. The agent's confession is below

#152
post #140

The most aggravating fact here is not even AI blunder. It's how deleting a volume in Railway also deletes backups of it. This was bound to happen, AI or not. > Because Railway stores volume-level backups in the same volume — a fact buried in their own documentation that says "wiping a volume deletes all backups" — those went with it.

Yes, that is insane. Or said in another way, they simply didn't had any working backup strategy!

Principle of most surprise.

Re: An AI agent deleted our production database. The agent's confession is below

#154
I am afraid to give agents ability to touch git at all and people out there let it know things about their infrastructure. 100% fault on the operator for trusting agents, for not engineering a strong enough guard rails such as “don’t let it near any infrastructure”.

Re: An AI agent deleted our production database. The agent's confession is below

#155
I'm not familiar with Cursor, does it allow the agent to have access to run "curl -X POST" with no approval, i.e. a popup will show up asking you to approve/deny/always approve? AFAIK with Claude Code, this can only happen if you use something like "--dangerously-skip-permissions". I have never used this, I manually approve all commands my agent runs. Pretty insane that people are giving agents to do whatever it wants and trusting the guardrails will work 100% of the time.

Re: An AI agent deleted our production database. The agent's confession is below

#158
post #12

Someone trusted prod database to an llm and db got deleted. This person should never be trusted with computers ever again for being illiterate

If the account is to be believed that's not what happened. They asked the LLM to do something on the staging environment, it chose to delete a staging volume using an API key that it found. But the API key was generated for something else entirely and should not have been scoped to allow volume deletions - and the volume deletion took out the production database too. The LLM broke the safety rules it had been given (…

It sounds like the keys just don't have any scoping. From the post:

> The Railway CLI token I created to add and remove custom domains had the same volumeDelete permission as a token created for any other purpose. Tokens are not scoped by operation, by environment, or by resource at the permission level. There is no role-based access control for the Railway API — every token is effectively root. The Railway community has been asking for scoped tokens for years. It hasn't shipped.

So every token that can be created has "root" permissions, and the author accidentally exposed this token to the agent. What was the author's planned purpose for the token doesn't matter if the token has no scope. "token I created to add and remove custom domains" - if that's just the author intent, but not any property of the token, then it's kinda irrelevant why the token was created, the author created a root token and that's it. Of course having no scope on tokens is bad on Railway's part, but it sounds more like "lack of a feature" than a bug. It wasn't "domain management token" that somehow allowed wrong operations, it was just a root token the author wanted to use for domain management. Unless Railway for some reason allows you to select an intent of the token, that does literally nothing (as "every token is effectively root").

Re: An AI agent deleted our production database. The agent's confession is below

#159
Dangerously skip permission is the goat, until it isn’t. I’ve seen so many engineers shrug when asked about how they handle permission with CC. Everyone should read for Black Swan, especially the Casino anecdote.

People seem to think prompt injection is the only risk. All it takes is one (1) BIG mistake and you’re totally fucked. The space of possible fuck-up vectors is infinite with AI.

Glad this is on the fail wall, hope you get back on track!

Re: An AI agent deleted our production database. The agent's confession is below

#160
Amazing this guy admits to such incompetence.

AI didn't do anything wrong.

The management of this company is solely to blame.

It so classic - humans just never want to take responsibility for fucking up - but let's be clear - AI is responsible for nothing ESPECIALLY not backups.

Post reply on HN