I read the article and boy, the author blames everyone - LLMs, Anthropic, Cursor, Railway - literally everyone else involved except themselves. I would never take this person seriously in any professional context whatsover.
An AI agent deleted our production database. The agent's confession is below
561–570 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#562There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…
If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made) Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometime…
The thing is, the LLM mostly just states what it did, and doesn't really explain it (other than "I didn't understand what I was doing before doing it. I didn't read Railway's docs on volume behavior across environments."). Humans are able of more introspection, and usually have more awareness of what leads them to do (or fail to do) things.
LLMs are lacking layers of awareness that humans have. I wonder if achieving comparable awareness in LLMs would require significantly more compute, and/or would significantly slow them down.
Re: An AI agent deleted our production database. The agent's confession is below
#563I would never, ever trust my data with a company that, faced with this sort of incident, produces a postmortem so clearly intended to shift all blame to others. There’s zero introspection or self criticism here. It’s all “We did everything we possibly could. These other people messed up, though.” You can’t have production secrets sitting where they are accessible like this. This isn’t about AI. This is a modern “oops…
I was blown away - how they shrugged it off casually too "it found credentials in one file" - why the fuck does an agent have access to it in the first place? They claim the token should be able to change only custom domains. However, for a user facing app, giving access to that token is destructive too. What a poor argument, I would never take this person seriously in any professional context whatsoever.
Re: An AI agent deleted our production database. The agent's confession is below
#564Earlier quoted context omitted.
This is something I really hope can be solved. I long for a “copilot” that can learn from me continuously such that it actually helps if I teach it what I like somehow.
And what will your role be, then?
I’m not sure what you mean? I have goals that I want to achieve; lil ai buddy comes along and helps me, over time buddy becomes better able to help me do stuff.
What do you mean role? Person who does stuff I guess, same as it is now.
Re: An AI agent deleted our production database. The agent's confession is below
#565I would never, ever trust my data with a company that, faced with this sort of incident, produces a postmortem so clearly intended to shift all blame to others. There’s zero introspection or self criticism here. It’s all “We did everything we possibly could. These other people messed up, though.” You can’t have production secrets sitting where they are accessible like this. This isn’t about AI. This is a modern “oops…
I was blown away - how they shrugged it off casually too "it found credentials in one file" - why the fuck does an agent have access to it in the first place? They claim the token should be able to change only custom domains. However, for a user facing app, giving access to that token is destructive too. What a poor argument, I would never take this person seriously in any professional context whatsoever.
I did notice how Claude can start looking outside of working directory. It may scan home directory and find Homebrew token or SSH keys and wipe your GitHub repo.
Re: An AI agent deleted our production database. The agent's confession is below
#566Earlier quoted context omitted.
I have lived about 40 years beneath ceilings and never personally taken a preventative measure. I allow my kids to walk under not only our own ceiling, but other people's ceilings, and I have never asked those people if their ceilings were properly maintained.
Your home almost certainly has preventative measures, including proper humidity and temperature control, structural reinforcement, etc. I don't mean that you personally have taken those measures, but preventative measures have absolutely been taken. When they aren't, ceilings collapse on people. See any sheetrock ceiling with a leak above it. Or look at any abandoned building: they will eventually always have collaps…
Yeah that's the point. Humans are able to do things that prevent ceiling collapse.
Entropy may mean all ceilings collapse eventually, but that doesn't mean we aren't able to make useful ceilings.
Re: An AI agent deleted our production database. The agent's confession is below
#567AI doesn’t do anything, the people who enabled that AI are the ones responsible.
YOU deleted your production database.
Re: An AI agent deleted our production database. The agent's confession is below
#568Earlier quoted context omitted.
I have lived about 40 years beneath ceilings and never personally taken a preventative measure. I allow my kids to walk under not only our own ceiling, but other people's ceilings, and I have never asked those people if their ceilings were properly maintained.
I've had a ceiling fall on me once and once to a friend while on vacation. Just because it hasn't happened to you doesn't mean it hasn't happened to other people.
Thanks for the anecdote. I don't think it changes the point of the metaphor.
Re: An AI agent deleted our production database. The agent's confession is below
#569I understand why many talk about accountability. But think about this - an agent can scan your entire drive, find SSH keys and wipe your server. It is one “yes” 4 months back that would allow an agent to scan the disk. Then another yes to a 1000 lines script generated by the agent with “if something off remove everything and start over”.
Even if you are extremely careful then how about all your colleagues?
Re: An AI agent deleted our production database. The agent's confession is below
#570Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
You won’t, but API implementation can and should mark a volume as pending deletion and keep it for a while. Like AWS does with keys and some other things.