AI didn't delete your database, you did • Hacker News Board

Re: AI didn't delete your database, you did

#161

Post by BadBadJellyBean » Tue, May 05, 2026, 4:26 PM UTC

mystraline wrote: post #155

BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> LLMs are a tool like every other. Only that it's non deterministic. If you stay away from the corporate SaaS token vendors, and run your own, you will find LLMs are deterministic, purely based on the exact phrase on input. And as long as the context window's tokens are the same, you will get the same output. The corporate vendors do tricks and swap models and play with inherent contexts from other chats. It makes o…

Yes and no. You might get the same output if you turn down the temperature, but you will probably not know the output without running it first. It's a bit like a hashing function. If I give the same input I get the same hash but I don't know which input will to which hash without running the function.

Also most LLMs are not run as I write a prompt and I will read output. Usually you have MCPs or other tools connected. These will change the input and it will probably lead to different outputs. Otherwise it wouldn't be a problem at all.

Re: AI didn't delete your database, you did

#162

Post by 6gvONxR4sf7o » Tue, May 05, 2026, 4:26 PM UTC

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Another view of the accountability is that we're currently often pointing accountability in the wrong direction, and it's gaining momentum. Aspects of it have been around so long it's a trope: important work around maintainability is undervalued.

Imagine two parallel universes:

- in one, you take ten minutes to make a dashboard that shows management what they asked for. It passes code review before merge and the exec who asked for it says it's what they wanted.

- in the other, you take a day or two to make it. Again, it passes code review before merge and the exec who asked for it says it's what they wanted.

Which version of you is more likely to get positive versus negative feedback? Even if the quick-to-build version isn't actually correct? If you're too slow and aren't doing enough that looks correct, you'll be held accountable. But if you're fast and do things that look correct but aren't, you won't be held accountable. You'll only be held accountable for incorrect work if the incorrectness is observed, which is rarer and rarer with fewer and fewer people directly observing anything.

So oddly, with nobody doing it on purpose, people get held accountable specifically for building things the way you're advocating.

I imagine that orgs that do lots of incorrect work could be outcompeted but won't be, because observability is hard and the "not get in trouble" move is to just not look too hard at what you're doing and move to the next ticket.

Re: AI didn't delete your database, you did

#163

Post by alansaber » Tue, May 05, 2026, 4:32 PM UTC

You didn't add a delete_database tool to your agent? You some kind of chicken?

Re: AI didn't delete your database, you did

#164

Post by sigbottle » Tue, May 05, 2026, 4:32 PM UTC

paroneayea wrote: post #128

sigbottle wrote: post #91

Earlier quoted context omitted.

About the blog you linked and not your comment: Doesn't symbolic AI have a lot of philosophical problems? Think back to Quine's two dogmas - you can't just say, "Let's understand the true meanings of these words and understand the proper mappings". There is no such thing as fixed meaning. I don't see how you get around that. Deep learning is admittedly an ugly solution, but it works better than symbolic AI at least.

Yes! But it's still valuable. How am I understanding your argument at all? I think my friend Jonathan Rees put it best: "Language is a continuous reverse engineering effort, where both sides are trying to figure out what the other side means." More on that: https://dustycloud.org/blog/identity-is-a-katamari/ This reverse engineering effort is important between you and me, in this exchange right here. It is a battle t…

I mean, Quine invented (the term) holism. I don't think we're on different pages. Maybe I should've specified a bit more what I was getting at.

This has very specific implications in symbolic ai specifically where historically the goal was mapping out the 'correct' representation of the space, then running formal analysis over it. That's why it's not a black box - you can trace out all of the steps. The issue is, is that symbolic AI just doesn't work. To my knowledge, as compared to all the DL wins we have.

I think the win of transformers proves that symbolic AI isn't the way. At the very least, the complex interactions that arise from in-context learning clearly in no way imply some fixed universal meaning for words, which is a big problem for symbolic AI.

Re: AI didn't delete your database, you did

#165

Post by bombcar » Tue, May 05, 2026, 4:32 PM UTC

BadBadJellyBean wrote: post #71

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> gparted wasn't to blame. I was.

These can both be true, especially if/when it has bad defaults. This is why you have things like "type the name of the database you're dropping" safety features - but you also have to name your production database something like "THE REAL DaTabaSe - FIRE ME" so you have to type that and not fall into the trap of ending up with the same name in test/development.

AI is particularly seductive because it sounds like a reasonable person has thought things out, but it's all just a giant confidence trick (that works most of the time, which makes it even more dangerous).

Re: AI didn't delete your database, you did

#166

Post by iamleppert » Tue, May 05, 2026, 4:33 PM UTC

This has been covered elsewhere, but if you swear at Claude Anthropic will automatically bump you down into a lower quality model. It was found in the recent source code leak of Claude Code. So that's probably what happened to the guy who's Cursor deleted his entire production database.

It just goes to show, if you're a jerk, expect to be treated like one (even by an AI model)! Be polite, people.

Re: AI didn't delete your database, you did

#167

Post by lelanthran » Tue, May 05, 2026, 4:37 PM UTC

pjc50 wrote: post #124

BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…

> This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder.

I point to the first USB port as the harbinger of things to come - try it one way, fail, turn it around, fail again, then turn it around one more time.

Just like AI, except there are unlimited axis upon which to turn it :-/

Re: AI didn't delete your database, you did

#168

Post by cik » Tue, May 05, 2026, 4:40 PM UTC

The whole life delete my database fiasco is being looked at the wrong way. Why did tooling have access to alter or drop? Why did tooling, in any way have more permissions than were m I nimallt necessary to do the job?

Decades ago we embraced POLA. What happened to basic hygiene? Sure the agent "screwed up", but it never should have had this access in the first place.

Re: AI didn't delete your database, you did

#169

Post by lelanthran » Tue, May 05, 2026, 4:41 PM UTC

BadBadJellyBean wrote: post #71

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was.

Much like how a poor workman always blames his tools, people using poor tools always blame themselves.

I mean, Donald E Norman wrote The Philosophy of Everyday Things in the 80s!(Later became "The Design of Everyday Things")

And yet, today, we will still have a bunch of people defending Gnome's design decisions, or the latest design decisions from Apple, etc.

Re: AI didn't delete your database, you did

#170

Post by kenjackson » Tue, May 05, 2026, 4:42 PM UTC

Barrin92 wrote: post #141

kenjackson wrote: post #131

Earlier quoted context omitted.

The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation, but it’s just something we cook up in our brain. Whether or not it’s the truth is far more complex. Oddly, despite LLMs being these huge networks with billions of parameters, we still probably do understand it better than we do our own brains.

>The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation Human brains and cognition do not work like LLMs, but that aside that's irrelevant. Existing machines can explain what they did, that's why we built them. As Dijkstra points out in his essay on 'the foolishness of natural language programming', the entire point of programming is: ( https://www.cs.ut…

That is absurd as a suggestion of it being the entire point of programming. In fact, it goes back to my original point - I have no idea why Djikstrs would say something so non-sensical, and likely neither did he.