[flagged]
An AI agent deleted our production database. The agent's confession is below
621–630 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#622I would never, ever trust my data with a company that, faced with this sort of incident, produces a postmortem so clearly intended to shift all blame to others. There’s zero introspection or self criticism here. It’s all “We did everything we possibly could. These other people messed up, though.” You can’t have production secrets sitting where they are accessible like this. This isn’t about AI. This is a modern “oops…
This was the line that did for me, as an old school backend engineer who has accidentally deleted way more production databases than I have fingers over the years -
> We have restored from a three-month-old backup.
You were absolutely screwed anyway if that was your backup strategy - deciding to plug your entire production infrastructure into a random number generator has only accelerated the process. Sort yourself out.
Re: An AI agent deleted our production database. The agent's confession is below
#623Earlier quoted context omitted.
The 'confession' is a CYA. Honestly the whole story doesn't really make sense - what's a "routine task in our staging environment" that needs a full-blown LLM? That sounds ridiculous to me. The takeaway is we commingled creds to our different environments, we gave an LLM access, and we had faulty backups. But it's totally not our fault.
Later they shift the blame to Railway for not having scoped creds and other guardrails. I am somewhat sympathetic to that, but they also violated the same rule they give to the agent - they didn't actually verify...
Sorry but are you implying that for every system you integrate with, you verify the scope of an API key by checking each CRUD operation on every API endpoint they provide?
Re: An AI agent deleted our production database. The agent's confession is below
#624Earlier quoted context omitted.
It's also important to realize that AI agents have no time preference. They could be reincarnated by alien archeologists a billion years from now and it would be the same as if a millisecond had passed. You, on the other hand, have to make payroll next week, and time is of the essence.
taps the "don't anthropomorphize the LLM" sign They don't have time preference because they don't have intent or reasoning. They can't be "reincarnated" because they're not sentient, they're a series of weights for probable next tokens.
An agent has more components than just an LLM, the same way a human brain has more components than just Broca's area.
Re: An AI agent deleted our production database. The agent's confession is below
#625Earlier quoted context omitted.
That’s not how language works, just how engineers think it works
This isn't a sarcastic response. What do you mean?
I just mean that the argument that words like “instructions”, “think”, “confess” are inaccurate when used in reference to a machine assumes that those words can only refer to humans/conscious beings, when really they can refer to more than that if used widely enough in those ways (in this case - text prediction following a human input).
So it’s not “anthropomorphizing” because when people use those words they don’t [typically] actually believe the machine can think or reason, it’s just the word that most closely matches the concept, it’s convenient. You’re extending the definition of the words to apply to non-conscious entities too, not applying consciousness to the entities.
It’s the same reason we call the handheld device we carry around to do everything a “phone” without a second thought. We don’t call it a phone because it’s primary purpose is calling, we call it a phone because the definition of the word “phone” has grown to include “navigates, entertains, takes pictures, etc”.
Re: An AI agent deleted our production database. The agent's confession is below
#626The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…
Yep. I made a "Read only" mode in pi by taking away "write" and "edit" tools. Claude Code used bash to make edits anyway.
Re: An AI agent deleted our production database. The agent's confession is below
#627Earlier quoted context omitted.
I was blown away - how they shrugged it off casually too "it found credentials in one file" - why the fuck does an agent have access to it in the first place? They claim the token should be able to change only custom domains. However, for a user facing app, giving access to that token is destructive too. What a poor argument, I would never take this person seriously in any professional context whatsoever.
I did notice how Claude can start looking outside of working directory. It may scan home directory and find Homebrew token or SSH keys and wipe your GitHub repo.
I do not use claude and will use agents only when I am forced to, so I'm genuinely asking here:
Can claude or other models not be run as a user or program with limited permissions? Do people just not bother to set it up? Why on earth would anyone run an RNG that can access $HOME/.ssh?
Re: An AI agent deleted our production database. The agent's confession is below
#628I would never, ever trust my data with a company that, faced with this sort of incident, produces a postmortem so clearly intended to shift all blame to others. There’s zero introspection or self criticism here. It’s all “We did everything we possibly could. These other people messed up, though.” You can’t have production secrets sitting where they are accessible like this. This isn’t about AI. This is a modern “oops…
It’s awful. "We had no clue this token had the permission to delete stuff!" - well buddy you issued it without deciding on permissions, it’s your job to assert that. Your latest recoverable backup is three months old? The rule is 3-2-1, you didn’t follow it. Nobody else to blame but yourself. And on and on he rambles…
But the database company (that he was trusting his customers' data with) hid how the database works in their docs! How could he have known!
Re: An AI agent deleted our production database. The agent's confession is below
#629>We have restored from a three-month-old backup
How is this not the first line in this article.
Mistakes happen. But not having automated backups ( weekly at a minimum, daily ideal ) is negligence. After looking at their website for a second, looks like they vibe coded large parts of their platform to rush to market.
PS: This is why developers need QA/Dev ops teams.
Re: An AI agent deleted our production database. The agent's confession is below
#630I use HITL AI dev tools all day long. As a human, I get to choose my stacks and my tools' agentic powers.
Seeing things like this, and the McDonald's support agent solving coding problems, I am now 95% over my imposter syndrome.