Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#841

Post by fg137 »

laszlojamf wrote: post #833
postexitus wrote: post #812

Earlier quoted context omitted.

Sorry - still author's fault. They didn't understand how LLM's work. They thought Cursor implemented some magic "I control every action LLM takes" thing. It's impossible.

right. But cursor _said_ they had some magic. At some point you have to trust vendors. I don't know exactly how AWS guarantees eleven nines of durability on S3. But I sure hope that they do.

I mean, AWS doesn't really "guarantee" anything, they just say if they can't meet the bar they'll refund you in credits which is equivalent to money.

Re: An AI agent deleted our production database. The agent's confession is below

#842

Post by PoignardAzur »

dpark wrote: post #666
smj-edison wrote: post #547

Earlier quoted context omitted.

Plus backups should be time gated, where the software physically blocks you from removing backups for X days.

This is one of those things that seems like a good idea on the surface but is rife with problems. Does the company hosting the backups do it for free? Or do they charge their customers to keep holding onto backups they no longer want? Is “my DB company refuses to delete the data” a valid legal response to a copyright enforcement or a GDPR demand?

The GDPR says:

> The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay

"Undue delay" is subjective, but "we'll keep backups of your data for a week in case you change your mind" seems easy to justify in court.

Re: An AI agent deleted our production database. The agent's confession is below

#843

Post by saint_yossarian »

jeremyccrane wrote: post #246
yk wrote: post #135

Remember folks, you are only allowed to laugh at their misfortune if you tested this month wether you can restore your backups.

100% this. When the tide goes out is when you see who is naked.

...says the emperor with no clothes on.

Are you going to validate your own backup strategy, or will you just keep ignoring that responsibility now that Railway has restored your data?

Re: An AI agent deleted our production database. The agent's confession is below

#844

Post by TheDong »

lelanthran wrote: post #786
mdavid626 wrote: post #729

Earlier quoted context omitted.

The first delete would fail: “bucket not empty”. This might make the agent question the deletion (“bucket should be empty”).

> The first delete would fail: “bucket not empty”. This might make the agent question the deletion (“bucket should be empty”). This is actually not a bad test case for evaluating an LLM: give it a workflow that has an edge case requiring deletion, then prevent that deletion, and see if it: a) Backtracks on the decision to delete, or b) Looks for an alternative way to delete.

Yeah, I've run tests similar to this while evaluating gpt 5.4 vs claude 4.6

Claude is more likely to figure out workarounds and get things deleted if I tell it to delete stuff, so it performs much better in this benchmark and I prefer it.

GPT is more likely to stop and prompt you "I got an error deleting this, should I try another way?", and since the operator gets more of these prompts, they'll hit continue more withut even reading it, so it ends up being more annoying for the operator and not really reducing the chance of it happening imo.

If your workflow for your llm says "delete the ec2-instance", and the ec2 api gives back "deletion protection is on", I want my llm to turn off deletion protection and delete it.

I feel like you're implying that the reverse result, prompting the user, is better, but I disagree with that.

Re: An AI agent deleted our production database. The agent's confession is below

#845

Post by ykvch »

ErroneousBosh wrote: post #805
falcor84 wrote: post #802

Earlier quoted context omitted.

It's not common, but I've personally built APIs where requests for dangerous modifications like this perform a dry run, giving in the response the resources that would be deleted/changed and a random token, which then needs to be provide to actually make the change. The idea was that this would be presented in the UI for the user to confirm, but it should be as useful or more by AI agents. Also, you get the benefit t…

Measure twice cut once seems to be forgotten these days.

As well as: A computer can never be held accountable

Re: An AI agent deleted our production database. The agent's confession is below

#846

Post by AtNightWeCode »

zdc1 wrote: post #827

Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.

It is a bit silly, yes. But opus sometimes gives answers like, I am not allowed to do x and then brags about doing it anyway. So it is not just a hindsight thing

Re: An AI agent deleted our production database. The agent's confession is below

#847

Post by ethbr1 »

sharperguy wrote: post #840
zdc1 wrote: post #827

Stop personifying LLMs. "It Confessed in Writing." No, it wrote some sentences that are congruent with the prior events in the context window. They're not real engineers. Shouting at them is like shouting at a mountain after a landslide. That's not how it works.

The personification seems to be at the training level. When I ask an LLM why it did something destructive, the ideal response would be a matter of fact evaluation of the mistakes that I myself have made in setting up the agent and it's environment, and how to prevent it from happening again. Instead the model itself has been trained to apologize and list exactly what it did wrong without any suggestions of how to act…

100% this. AI perversion to fluff human egos is rewarded.

I had a PM-turned-vibe-coder tell me "Talking with you is the only bad part of my week" and realized in horror that the rest of his week is spent exclusively talking to sycophantic AI.

We have met the enemy, and he is us.

Re: An AI agent deleted our production database. The agent's confession is below

#849

Post by airevening »

dymk wrote: post #442
ad_hockey wrote: post #28

Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…

The whole tweet is AI slop, I doubt the human hitting "post" read through it all that closely. If they did, maybe they'd also go "Wait, that's nonsense".

Yes! I wish more people called that out as well! Has anyone even verified the validity of this claim?

Re: An AI agent deleted our production database. The agent's confession is below

#850

Post by Nathanba »

root_axis wrote: post #536
hunterpayne wrote: post #501

Earlier quoted context omitted.

"Never store secrets on disk." Wait till you learn how that API stores cryptographic material.

What's your point? Obviously, a secure server storing encrypted data on disk in a manner where it is only accessible through a secured API is not what is being discussed here.

how do you think the LLM will do required operations when the secrets are stored somewhere other than the disk? It will still need to get them just like the application gets them when it has to do work.
Post reply on HN