Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent deleted our production database. The agent's confession is below

twitter.com

Post reply on HN
Print view View original

201–210 of 1001 posts

Re: An AI agent deleted our production database. The agent's confession is below

#201

Post by gus_massa »

ad_hockey wrote: post #28

Minor point, but one of the complaints is a bit odd: > curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing. It's an API. Where would you type DELETE to confirm? Are the…

Assuming the API has some secret spot to write DELETE, wouldn't the chatbot just send DELETE and make the protection only delay the disaster for 10 seconds?

Re: An AI agent deleted our production database. The agent's confession is below

#202

Post by elliotpage »

Quarrelsome wrote: post #197
Lionga wrote: post #177

Earlier quoted context omitted.

The most aggravating fact is that the AI slopper that got owned by his dumbness and AI just post an AI generated post that will generate nothing but schadenfreude

its much more aggravating that it looks like they're learning nothing by pushing blame onto everything else except themselves.

I'm glad that I'm not the only person who felt this! It does feel like the post is missing some deserved self-reflection.

Re: An AI agent deleted our production database. The agent's confession is below

#203

Post by fc417fc802 »

jmalicki wrote: post #122
pierrekin wrote: post #103

Earlier quoted context omitted.

I agree that the model can help troubleshoot and debug itself. I argue that the model has no access to its thoughts at the time. Split brain experiments notwithstanding I believe that I can remember what my faulty assumptions were when I did something. If you ask a model “why did you do that” it is literally not the same “brain instance” anymore and it can only create reasons retroactively based on whatever context i…

It does have access to its thoughts. This is literally what thinking models do. They write out thoughts to a scratch pad (which you can see!) and use that as part of the prompt.

It's important to be aware that while those "thoughts" can be a useful aid for human understanding they don't seem to reliably reflect what's going on under the hood. There are various academic papers on the matter or you can closely inspect the traces of a more logically oriented question for yourself and spot impossible inconsistencies.

Re: An AI agent deleted our production database. The agent's confession is below

#207

Post by pixl97 »

cmiles74 wrote: post #190
vidarh wrote: post #93

Earlier quoted context omitted.

If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made) Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometime…

None of the developers that I’ve worked with have had the hemispheres of their brains severed. I suspect this is pretty rare in the field.

This still doesnt stop post ad hoc explanations by humans.

Re: An AI agent deleted our production database. The agent's confession is below

#209

Post by filoleg »

alastairr wrote: post #25

If it's real this is a terrible thing to have happen. However the moral of this story is nothing to do with AI and everything to do with boring stuff like access management.

^This.

One of the top replies on twitter to the OP can be boiled down to "you treat AI as a junior dev. Why would you give anyone, let alone a junior dev, direct access to your prod db?"

And yeah, I fully agree with this. It has been pretty much the general consensus at any company I worked at, that no person should have individual access to mess with prod directly (outside of emergency types of situations, which have plenty of safeguards, e.g., multi-user approvals, dry runs, etc.).

I thought it was a universally accepted opinion on HN that if an intern manages to crash prod all on their own, it is ultimately not their fault, but fault of the organizational processes that let it happen in the first place. It became nearly a trope at this point. And I, at least personally, don't treat the situation in the OP as anything but a very similar type of a scenario.

Post reply on HN