Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

281–290 of 329 posts

Re: AI didn't delete your database, you did

#281

Post by Melatonic »

stingraycharles wrote: post #35
edot wrote: post #20

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissions would also blame interns for deleting production whatever. Blame should go up, praise should go down. People always invert these.

> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the faul…

I believe they were being sarcastic and saying the same thing as you :-)

Re: AI didn't delete your database, you did

#282

Post by clickety_clack »

mattgreenrocks wrote: post #272
clickety_clack wrote: post #192

Earlier quoted context omitted.

A band saw is always a screaming band of bladed death. An LLM is sometimes a buddy, sometimes a mentor, and only sometimes a guy that drops your database.

Maybe we can just not give it access to production databases ever? Not picking on you, but AI maximalism has infected tech to the point where we talk about how to stop AI from deleting prod instead of seeing that giving AI access to prod is a foolish idea to begin with.

I mean that it’s easy to be careful around a bandsaw because it’s clearly dangerous. The danger with LLMs is that they don’t seem overtly dangerous so you just go right ahead and throw your whole arm in there.

Re: AI didn't delete your database, you did

#283

Post by lowbloodsugar »

Joker_vD wrote: post #275
lowbloodsugar wrote: post #250

Earlier quoted context omitted.

People make bad decisions all the time . Insanity is not required. My remark was pointing out the larger failure mode of people acting contrary to the good of the team for personal gain, eg creating a problem and blaming someone else to reduce their chances of competing for a promotion. But to your point, an SDE doesn’t need to be insane to bypass a 2 PR and force a change into production. They just need to be panick…

> They just need to be panicked, or overconfident, or overworked. One of the best thing about digital computers, compared to humans, was that they can't be the first or the third thing you mentioned; unfortunately, they absolutely are the second ("the machine does exactly what you told it to do, not what you want it to do"), and at inhuman speeds. Presumably, AI would (need, actually — Nick Bostrom puts a fairly reas…

Computers absolutely can be overworked. Plenty of outages caused by system overloads. Or a system deletes a file because it believes it to be no longer in use but only because some queue was full. I’m not arguing that it’s not a problem because humans have the same problem. Part of my job is making sure humans can’t fuck it up either. I’m saying “assume the worst” and make sure the processes catch human and AI mistakes.

Also, I think Nick makes the same point as me: AI will attempt to kill us.

Re: AI didn't delete your database, you did

#284

Post by kenjackson »

Barrin92 wrote: post #182
kenjackson wrote: post #170

Earlier quoted context omitted.

That is absurd as a suggestion of it being the entire point of programming. In fact, it goes back to my original point - I have no idea why Djikstrs would say something so non-sensical, and likely neither did he.

what do you mean "likely neither did he", I literally linked you the piece in which he said it. And of course he of all people would make that (correct) point, because he was always the strongest advocate of the virtue of formal correctness of programming languages, again from his article: "A short look at the history of mathematics shows how justified this challenge is. Greek mathematics got stuck because it remaine…

"what do you mean "likely neither did he", I literally linked you the piece in which he said it."

Saying an explanation it and actually knowing why you did it are two different things. That's exactly my point.

And then Boole's quote -- good quote, but I think you (not Boole) are conflating precision with motivation.

Re: AI didn't delete your database, you did

#285

Post by mattgreenrocks »

clickety_clack wrote: post #282
mattgreenrocks wrote: post #272

Earlier quoted context omitted.

Maybe we can just not give it access to production databases ever? Not picking on you, but AI maximalism has infected tech to the point where we talk about how to stop AI from deleting prod instead of seeing that giving AI access to prod is a foolish idea to begin with.

I mean that it’s easy to be careful around a bandsaw because it’s clearly dangerous. The danger with LLMs is that they don’t seem overtly dangerous so you just go right ahead and throw your whole arm in there.

That clarifies it...thanks!

Re: AI didn't delete your database, you did

#286

Post by pier25 »

Sohcahtoa82 wrote: post #278
pier25 wrote: post #260

Earlier quoted context omitted.

> LLMs are non-deterministic For someone who complains about a lack of nuance it's surprising you're completely missing my point. The lack of trust is precisely my point. Either AI companies are made accountable for providing un unreliable service or they need to stop selling and marketing these LLMs as if they were infallible.

Three people responded to you, and it seems that all three missed the point you were trying to make. Maybe you should change your argument style and actually articulate a point rather than taking the insufferable approach of asking a "gotcha" question that ends up getting misunderstood. If it's one person misunderstanding you, then its on them. If everyone is misunderstanding you, then it's on you.

Attacking someone is probably the worst version of the argument you could make.

Re: AI didn't delete your database, you did

#287

Post by bsoles »

BadBadJellyBean wrote: post #71
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> gparted wasn't to blame. I was.

I don't know about gparted, but I always felt that "rm -i" should have been the default. The safe option should always be the default and you can optionally make it unsafe. Same goes with "mv -i".

Re: AI didn't delete your database, you did

#288

Post by handoflixue »

Barrin92 wrote: post #141
kenjackson wrote: post #131

Earlier quoted context omitted.

The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation, but it’s just something we cook up in our brain. Whether or not it’s the truth is far more complex. Oddly, despite LLMs being these huge networks with billions of parameters, we still probably do understand it better than we do our own brains.

>The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation Human brains and cognition do not work like LLMs, but that aside that's irrelevant. Existing machines can explain what they did, that's why we built them. As Dijkstra points out in his essay on 'the foolishness of natural language programming', the entire point of programming is: ( https://www.cs.ut…

Okay, so you've established that LLMs aren't programming. They are unlike existing machines. The closest analogy we have for them is the human brain, which also seems to include a lot of neural net architecture.

Now, physics says that everything can be explained mathematically, including the human brain. Obviously, on some level, an LLM can be explained. But despite hundreds of years of science, we still don't understand the human brain. Some systems are just really complex and difficult to understand.

Given all of that, I see no reason to assume that we'll be able to understand LLMs anytime soon. Especially given we keep growing more complex ones.

Re: AI didn't delete your database, you did

#289

Post by nulltrace »

ebiester wrote: post #77

First, no matter what you do, if a human has write access to the production database, the database can be deleted. Second, there is a legitimate reason to destroy a database in development and automation. The biggest problem I see is often treating your development data like pets not cattle. You absolutely need to have safeguards that this cannot be run in production, but if a human has access to the credentials to r…

Most IAM policies start as "whatever made the deploy pass." Need rds:CreateDBInstance? Fine, rds:* it is. Ship it. Months later that same role can wipe the cluster and nobody remembers why it ever had that permission.

Separate accounts help, but only if someone actually goes back and cleans it up, which… yeah, doesn't really happen.

Post reply on HN