Learn to code yourself, stop using slop generators, then shit like this doesn't happen.
Senior software dev brother :)
[dead]
An AI agent deleted our production database. The agent's confession is below
851–860 of 1001 posts
Re: An AI agent deleted our production database. The agent's confession is below
#852Earlier quoted context omitted.
I don't think this is a minor point. It seems clear by this point that the author is clueless how even API works and are just trying to shift blame for third-parties instead assuming that they're just vibecoding their whole product without doing proper checks. Yes sure, there seems to be lots of ways this issue could have been mitigated, but as other comments said, this mostly happened because the author didn't do it…
He also seems to be lying, he wrote on Twitter the agent was in plan mode. That part has to be exaggerated.
I can’t say for sure, but I think Claude’s mode is nothing more than part of the system prompt. I don’t think it actually takes away web request or file write tools. I say this because I could swear I’ve seen Claude go ahead and make some changes even while we’re in plan mode. Web requests certainly, because it can fetch docs and so forth.
Re: An AI agent deleted our production database. The agent's confession is below
#853Not at all surprising this happened. Stop vibe coding if you value your business/customers.
Every senior/principal developer worth his/her salt knows how bad AI still is when it comes to coding.
DO. NOT. BELIEVE. AI. CEOS.
Do not hand over control of your production data/services to AI. No matter how you might feel you are missing out. Your feelings are not > your customers.
Value your customers. They are your bread and butter. Not AI CEOs or AI bros who want to sell you shovels in this inane fake gold rush.
Re: An AI agent deleted our production database. The agent's confession is below
#854The only healthy stance you should have on AI Safety: If AI is physically capable of misbehaving, it might ($$1), and you cannot "blame" the AI for misbehaving in much the same way you cannot blame a tractor for tilling over a groundhog's den. > The agent's confession After the deletion, I asked the agent why it did it. This is what it wrote back, verbatim: Anyone who would follow a mistake like that up with demandin…
> Anyone who would follow a mistake like that up with demanding a confession out of the agent is not mature enough to be using these tools. Anyone like that is not mature enough to be managing humans. I'm glad that these AI tools exist as a harmless alternative that reduces the risk they'll ever do so.
When I read the title I expected some kind of satire. I wonder if author considered giving the AI a penance.
Maybe if it wrote "I will not delete production database again" a million times, it would prevent such situations in future?
Re: An AI agent deleted our production database. The agent's confession is below
#855Earlier quoted context omitted.
I suspect we are not describing the same thing. When a human asks another human “why did you do X?”, the other human can of course attempt to recall the literal thoughts they had while they did X (which I would agree with you are quite analogous to the LLMs chain of thought). But they can do something beyond that, which is to reason about why they may have the beliefs that they had. “Why did you run that command?” “B…
> Whereas for an agent it will happily include details that are not literally in its chain of thought as justifications for its decisions. Humans do this too, ALL THE TIME. We rationalize decisions after we make them, and truly believe that is why we made the decision. We do it for all sorts of reasons, from protecting our ego to simply needing to fill in gaps in our memory. Honestly, I feel like asking an AI it’s tr…
Humans don't do this all the time. I think you are conflating things to further this false idea that there is no distance between human thinking and the behavior of LLMs. The kind of rationalization humans sometimes do generally happens over a period of time. Humans are also not "rationalizing" their actions all the time. Also, when humans do what you call "rationalizing," it is to serve some kind of interest, beyond responding to a prompt.
Re: An AI agent deleted our production database. The agent's confession is below
#856Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.
You forget that people running these companies have near zero understanding of what LLM is and rely solely on their personal experience and social media hype.
I've inclined to believe that they also have outsourced their thinking process to Agents. It's useless trying to talk sense into them. Let them crash and burn. And pray there will be something left working, after all this madness ends.
Re: An AI agent deleted our production database. The agent's confession is below
#857Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.
I agree with you but I feel like this piece is meant to be a cautionary tale to CEOs and the like to not consider them as real engineers.
Re: An AI agent deleted our production database. The agent's confession is below
#858My dad always said "pedestrians have the right of way" every time one crossed the street, but wouldn't let us cross the street when the pedestrian light came on until the cars stopped. When I repeated his rule back to him, he said "you may have the right of way, but you'll still be dead if one hits you". My adult synthesis of this is "it's fine to do something risky, as long as you are willing to take the consequence…
This kind of is Postel's law, in a way:
Re: An AI agent deleted our production database. The agent's confession is below
#859Giving an agent this level of access to infra is doing a disservice to people who've trusted this guy with their business.
Re: An AI agent deleted our production database. The agent's confession is below
#860Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…
Guys, did you bother checking the poster's profile? https://xcancel.com/lifeof_jer. SEE THE TWEET BELOW. Smells like a ragebait post to me. Also search online for his alleged "PocketOS" company with software for car rental businesses. I couldn't find anything on Google. (Of course, I might be wrong)
"The future of SEO is AIO" https://xcancel.com/lifeof_jer/status/2034409722624061772 March 18