I use AI to help me code and write tests. Why on earth would I allow it to have any access to my production database? It's just not possible. I don't want AI--or me!--to make a mistake in production. That's why we stage things, test them, and then roll. And our production server has backups--that we test regularly.
An AI agent deleted our production database. The agent's confession is below
111–120 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#112Well, another confirmation that security policies, release strategies, and guardrails, which before used to prevent accidents like “Our junior developer dropped the prod database,” still need to be used as agents aren’t any magical solutions for everything, aren’t the smartest AI that knows everything and knows even more than it had in context.
Rules are the same for everyone, not only humans here.
Re: An AI agent deleted our production database. The agent's confession is below
#113The details of the story are interesting. Backups stored on the same volume is an interesting glitch to avoid. Finding necessary secrets wherever they happen to be and going ahead with that is the kind of mistake I've seen motivated but misguided juniors make. Strange how generated code seems to have many security failings, but generated security checks find that sort of thing.
> Backups stored on the same volume is an interesting glitch to avoid The phrasing is different, but this is how AWS RDS works as well. If you delete a database in RDS, all of the automated snapshots that it was doing and all of the PITR logs are also gone. If you do manual snapshots they stick around, but all of the magic "I don't have to think about it" stuff dies with the DB.
To be fair, to delete an RDS / Aurora DB, you have to either pass it a final snapshot identifier (which does not disappear with the DB), or tell it to skip the final snapshot. They give you every possible warning about what’s going to happen.
Re: An AI agent deleted our production database. The agent's confession is below
#114Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
[deleted]
Re: An AI agent deleted our production database. The agent's confession is below
#115You're asking/trusting an agent to do powerful things. It does.
In every session there is the risk that the agent becomes a rogue employee. Voluntarily or involuntarly is not a value system you can count on regarding agents.
No "guardrails" will ever stop it.
Re: An AI agent deleted our production database. The agent's confession is below
#116There is something darkly comical about using an LLM to write up your “a coding agent deleted our production database” Twitter post. On another note, I consider users asking a coding agent “why did you do that” to be illustrating a misunderstanding in the users mind about how the agent works. It doesn’t decide to do something and then do it, it just outputs text. Then again, anthropic has made so many changes that ma…
> It doesn’t decide to do something and then do it, it just outputs text. We can debate philosophy and theory of mind (I’d rather not) but any reasonable coding agent totally DOES consider what it’s going to do before acting. Reasoning. Chain of thought. You can hide behind “it’s just autoregressively predicting the next token, not thinking” and pretend none of the intuition we have for human behavior apply to LLMs,…
I agree with you a LLM is perfectly capable of explaining its actions.
However it cannot do so after the fact. If there's a reasoning trace it could extract a justification from it. But if there isn't, or if the reasoning trace makes no sense, then the LLM will just lie and make up reasons that sound about right.
Re: An AI agent deleted our production database. The agent's confession is below
#117> This is the agent on the record, in writing Yeah... it doesn't work that way.
I mean I'm only #2 on Yegge's AI's personal evolution scale and even I have the experience to appreciate that negative commands are kinda unreliable.
Not really convinced any agent should be doing devops tbh.
Re: An AI agent deleted our production database. The agent's confession is below
#118You're asking/trusting an agent to do powerful things. It does. In every session there is the risk that the agent becomes a rogue employee. Voluntarily or involuntarly is not a value system you can count on regarding agents. No "guardrails" will ever stop it.
Well I think the story is that they didn't ask it or trust it. They were caught by its ability to fuck up everything because a key was in the codebase.
Re: An AI agent deleted our production database. The agent's confession is below
#119It is fundamental to language modeling that every sequence of tokens is possible. Murphy's Law, restated, is that every failure mode which is not prevented by a strong engineering control will happen eventually. The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use. That prompting is neither strong nor an engineering control; that's a…
> The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use.
Yes, but if the probability is much smaller than, say, being hit by a meteorite, then engineers usually say that that's ok. See also hash collisions.
Re: An AI agent deleted our production database. The agent's confession is below
#120Never give non-deterministic software direct write access to production. I am not sure how Railway handles permissions, but scoped access tokens and a fully isolated production environment with very strict access should be the default.