[dead]
An AI agent deleted our production database. The agent's confession is below
811–820 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#812Interesting story. But despite Cursors or Railways failure, the blame is entirely on the author. They decided to run agents. They didnt check how Railway works. They relied on frontier tech to ship faster becsuse YOLO. I really feel sorry for them, I do. But the whole tone of the post is: Cursor screwed it up, Railway screwed it up, their CEO doesnt respond etc etc. Its on you guys! My learning: Live on the cutting e…
I get what your saying, but this is resonating with me and making me feel for the author: Cursor: we have top notch safeguards for destructive operations, you have our guarantee, we are the best Author: uses their tools expecting their guarantees to be true (I would expect them to have a confirmation before destructive operation outside their prompt, as a coded system guardrail) Cursor AI: Does destructive operation…
Sorry - still author's fault. They didn't understand how LLM's work. They thought Cursor implemented some magic "I control every action LLM takes" thing. It's impossible.
Re: An AI agent deleted our production database. The agent's confession is below
#813Im really tired of people saying "the agent did this" or posting agents excuses as if they still think agents behaviour is a safety layer not a mere usability tool. Posts like this reinforce this misunderstanding in juniors instead of learning to focus on the workflows and tools. "well, you should have used a better model." >> this is nothing any sane person even remotely knowledgable will ever say. Non deterministic systems gonna nondeterminist so what? The issue is relying on cli/imperative tools and seeing manual changes to subdomains as a casual, when in reality there are a lot of implications on changing your domains (or anything about your hosting setup), this should be completely automatic and the system to do this needs be driven by gitops with declarative truth, you know the things the devops world has been perfecting and saying for the past 10 years?
The only missing interesting thing is: did this token file live inside the current project folder? Or did cursor fully fail to constrain actions to the sane default? In either case i make a strong point to disallow agents accessing any git ignored files even if inside the folder, this will prevent a whole breadth of similar problems, with minimal downside, plus you can always opt subsets of ignores back in where it makes sense.
One last point i want to make is do not trust just your agent harness, if it matters at least require one or more layers of safety around the harness. Use sandboxes or runtime enforcement of rules. Do not accumulate state there but use fresh environments for every session. This will reduce the risk for things like this happening by an order of magnitude.
Re: An AI agent deleted our production database. The agent's confession is below
#814> Read that again. The agent itself enumerates the safety rules it was given and admits to violating every one. This is not me speculating about agent failure modes. This is the agent on the record, in writing. Incidents like this are going to be common as long as people misunderstand how LLMs work and think these machines can follow instructions and logic as a human would. Even the incident response betrays a fundam…
"I literally requested no screw ups, and this is a screw up" I bet these people are bad at managing humans too.
Maybe - humans have agency, they understand actions / consequences.
AI agents do not have agency(!), they have no understanding of consequences. They actually have no understanding. At all.
Re: An AI agent deleted our production database. The agent's confession is below
#815Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
I've sometimes seen a variable like "areyousure" which needs to be set to true. Sometimes there's a force flag. And "agree to eula" fields are somewhat common.
Re: An AI agent deleted our production database. The agent's confession is below
#816Earlier quoted context omitted.
Plus backups should be time gated, where the software physically blocks you from removing backups for X days.
This is one of those things that seems like a good idea on the surface but is rife with problems. Does the company hosting the backups do it for free? Or do they charge their customers to keep holding onto backups they no longer want? Is “my DB company refuses to delete the data” a valid legal response to a copyright enforcement or a GDPR demand?
I have no idea about the former but yes, it is a valid excuse for latter. Ok, maybe not that specific one but in general backups are going to be excluded, especially those stored on tapes or WORM media - no one expects company to remove offending record here and now, as long it is inaccessible for all practical purposes.
Re: An AI agent deleted our production database. The agent's confession is below
#817Can we please stop anthropomorphizing LLMs? It is extremely unhealthy and seems like it feeds into people's irresponsible use of a tool that could otherwise be useful if we stopped treating prediction machines like what they are not.
Re: An AI agent deleted our production database. The agent's confession is below
#818The good news is he learned his lesson by having his hosting provider recover his production data, no need for backups ever again. https://x.com/lifeof_jer/status/2048576568109527407
> Now let's work together and improve the tooling at Railway b/c I have always LOVED the service stack and tooling
He learned NOTHING, that is my take. If he learned something it would be to have people that know how their provider works, that know how their API tokens work and above all to have people - starting with him - that acknowledge their mistakes so that they learn from them!
Re: An AI agent deleted our production database. The agent's confession is below
#819they allowed ai agent read write on prod db. the confession is above
Re: An AI agent deleted our production database. The agent's confession is below
#820Earlier quoted context omitted.
I agree that this is the author’s fault considerably more than it is Railway’s, however I have learned from experience that no matter how many “are you sure you want to do this” prompts you have, sometimes users delete stuff they didn’t intend to delete and it’s better to not delete immediately but put it in a queue for deletion in a few hours and offer a way to reverse it. Even if it’s 100% user error, the user is v…
Funny how he points the finger at everyone but himself.
the kind of attitude you really need to get your agents to delete your prod lol