AI didn't delete your database, you did • Hacker News Board

Re: AI didn't delete your database, you did

#201

Post by polytely » Tue, May 05, 2026, 5:22 PM UTC

pjc50 wrote: post #124

BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…

This is so well put, and it not only happens on the user level but also on the organisational level. Where you can completely abdicate both responsibility and explanation by moving the complicated questions into the black box of an AI model.

Re: AI didn't delete your database, you did

#202

Post by Apocryphon » Tue, May 05, 2026, 5:23 PM UTC

JamesSwift wrote: post #191

figmert wrote: post #41

Earlier quoted context omitted.

> The author is rather vague about what exactly it asked it to do, he just says there was a “credentials mismatch” and Claude took the initiative to fix it by deleting the volume. But it’s likely that they are somewhat downplaying their culpability by being vague. I was just talking to my girlfriend saying I've realised that I've not written a single line of code, nor have I debugged myself for at least the past 3 mo…

Give an agent an obstacle and it will try to find a way around it. Most of the egregious commands Ive seen it run were fundamentally due to something blocking it from accomplishing a task. So eg if you block network access for the agent, you will get all sorts of creative solutions to try and get around the problem. This is also why its nearly impossible to corral commands. Because eventually it will rot13 encode a s…

Paperclip maximizer!

Re: AI didn't delete your database, you did

#203

Post by BadBadJellyBean » Tue, May 05, 2026, 5:25 PM UTC

skydhash wrote: post #181

BadBadJellyBean wrote: post #145

Earlier quoted context omitted.

I agree that LLMs could be more open about their dangers and that people are bad at judging risks sometimes. Still I think a band saw has very little warning on it and by it's design there is very little anyone can do about me cutting off my finger if I am not careful. LLM companies can do very little about the unpredictability of LLMs. So we have to choose how for we will let it go. In the end the LLM only produces…

It’s possible to make AI safe, but that also throws most of the gains out of the windows, especially if the artifact is a diff which can take time to review. In IT, you often have to give access to possible malicious users, you just have to scope what they can do. But the push is agentic everything, where AI needs to be everywhere, not in its own sandbox.

We don't have to blindly follow every trend. If agentic is not safe then it's on me if I use it and something breaks.

Re: AI didn't delete your database, you did

#204

Post by BadBadJellyBean » Tue, May 05, 2026, 5:27 PM UTC

lelanthran wrote: post #169

BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Much like how a poor workman always blames his tools, people using poor tools always blame themselves. I mean, Donald E Norman wrote The Philosophy of Everyday Things in the 80s! (Later became "The Design of Everyday Things") And yet, today, we will still have a bunch of people defending Gnome's design…

I am still to blame if I choose a bad tool. Especially if I should know better.

Re: AI didn't delete your database, you did

#205

Post by 52-6F-62 » Tue, May 05, 2026, 5:28 PM UTC

BadBadJellyBean wrote: post #156

locknitpicker wrote: post #144

Earlier quoted context omitted.

> - a tool is activated with operator intent (at some point in the call-chain) This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing. Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing th…

> if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system? Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms. Of course everyone is differently qualified to do this but my argument is more about professionals. Managers…

Except what we have here is razor blade companies getting the government to heavily subsidize present razor blade production running massive advertising campaigns and intense intra-industry pressure to give said razor blades to babies under fear of losing your job or "falling behind" those not giving razor blades to babies.

Let's not forget all the razor blade enthusiasts just screaming at you that you are using babies with razor blades wrong and that it works totally fine for them.

Re: AI didn't delete your database, you did

#206

Post by kbrkbr » Tue, May 05, 2026, 5:31 PM UTC

armada651 wrote: post #157

kbrkbr wrote: post #78

Earlier quoted context omitted.

An LLM generates plausible text token by token. It is at its core a deterministic function with some randomization and some clever tricks to make it look like an agent dialoguing or reasoning. Plausible text sometimes is right, sometimes not. Humans have a world model, a model of what happens. LLMs have a model of what humans would plausibly say. The only good guardrail seems human-in-the-loop.

This is such a motte-and-bailey argument. Whenever people point out LLMs aren't actually intelligent then you're an anti-AI Luddite. But whenever an AI does something catastrophically dumb it's absolved of all responsibility because "it's just predicting the next token". I'm getting so tired of this.

I think they are not actually intelligent. Fix all random seeds and other sources of randomness, and try the same prompt twice, and check how intelligent that looks, as a first approximation.

On a more technical level very serious people have voiced doubts, for example Richard Sutton in an interview with Dwarkash Patel [1].

[1] https://m.youtube.com/watch?v=21EYKqUsPfg&pp=ygUnZmF0aGVyIG9...

Re: AI didn't delete your database, you did

#207

Post by Joker_vD » Tue, May 05, 2026, 5:33 PM UTC

lowbloodsugar wrote: post #196

yubblegum wrote: post #120

Earlier quoted context omitted.

> Humans aren’t any better We're different. People have fairly consistent faults. LLMs are nondeterministic even in terms of how they fail. A high value human resource can be counted on to deliver. That, imho, is in fact one of the primary roles of good management: putting the right person in the appropriate position. Process engineering has worked to date because both the human and mechanical components of a system…

> A high value human resource can be counted on to deliver. Anyone who has encountered politics, psychopaths and narcissists knows that this isn’t always true.

Normally, people don't suddenly go insane, snap and start deliberately deleting things in production. Sure, it happens, but very, very rarely.

Re: AI didn't delete your database, you did

#208

Post by stavros » Tue, May 05, 2026, 5:35 PM UTC

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

I feel like "AI didn't delete your database, you did" is all about who has accountability, though.

Re: AI didn't delete your database, you did

#209

Post by davidatbu » Tue, May 05, 2026, 5:38 PM UTC

HarHarVeryFunny wrote: post #176

Yes, of course any company is responsible for what they ship, regardless of what tools were used to develop it. However, at least in the US, it is usual for companies to advise against use of their products in a way that may cause harm, and we certainly don't see that from the LLM vendors. We see them claim the tech to be near human level, capable of replacing human software developers (a job that requires extreme re…

Well, of the top of my head, both chatgpt.com and Gemini have text on their home page to the effect of "AI can make mistakes". I'll bet a few bucks such copy can be found in other places, including the terms of service.

Re: AI didn't delete your database, you did

#210

Post by traderj0e » Tue, May 05, 2026, 5:42 PM UTC

MarleTangible wrote: post #7

gowld wrote: post #4

Distinction without a difference.

I think it's about owning the consequences of one's own actions.

Yeah but it's funnier to blame the AI. And when the "real coders don't use AI" people get pissed off at the joke, you double down instead.