Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

91–100 of 329 posts

Re: AI didn't delete your database, you did

#91

Post by sigbottle »

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

About the blog you linked and not your comment:

Doesn't symbolic AI have a lot of philosophical problems? Think back to Quine's two dogmas - you can't just say, "Let's understand the true meanings of these words and understand the proper mappings". There is no such thing as fixed meaning. I don't see how you get around that.

Deep learning is admittedly an ugly solution, but it works better than symbolic AI at least.

Re: AI didn't delete your database, you did

#92

Post by sam0x17 »

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

There used to be a lot of research into using deep NNs to train decision trees, which are themselves much less of a black box and can actually be reasoned about. I wonder where that all went?

Re: AI didn't delete your database, you did

#93

Post by Romario77 »

stingraycharles wrote: post #35
edot wrote: post #20

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissions would also blame interns for deleting production whatever. Blame should go up, praise should go down. People always invert these.

> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the faul…

If you read what happened it's not that cut&dry. Railway (their cloud provider) gave them a token for operations. The AI was working on staging at the moment. Since the token had wide range permissions AI used it in it's routine operations to delete a volume to fix something and this resulted in their prod and backup data deletion.

So, here at least some of the blame belongs to Railway - how they organized their security, how the volume deletion deletes backups as well.

They since fixed some of these issues, so a similar mistake from someone won't be as catastrophic.

Re: AI didn't delete your database, you did

#94

Post by jmalicki »

madeofpalk wrote: post #81
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

People are eschewing their own accountability, blaming the tools instead for their poor decision making and lack of access controls. Why is it possible for you to fat-finger your way to deleting production database locally?

Some AI systems have done things like hack out of a docker container to access correct answers while being benchmarked.

That is mildly concerning and I will give holding the AI accountable to some degree when it is actively being malicious like that, even though the user could have locked things down even more.

But it had write access to the prod DB without circumventing controls and dropped your tables? That is just a total fail.

Re: AI didn't delete your database, you did

#95

Post by QuantumNomad_ »

2ndorderthought wrote: post #75
cheschire wrote: post #67

Earlier quoted context omitted.

I don’t think it’s missing, I just think it’s seen as a liability, and American society has been known to absolutely obliterate people who are liable. Everyone thinks they have the right to judge, and use the massive amounts of available information to do so, even if they haven’t been trained to judge.

List the companies who received a fine worthy of the damage they caused in recent history. List the ones who didn't. It's not about judging. We are socializing the losses to the public and capitalizing the profits for the already wealthy.

We dont know the final amount, as they settled out of court, but in 1992 a woman was awarded hundreds of thousands of dollars by the judge after receiving third degree burns from a coffee at a McDonalds.

She had originally asked for $20,000 to cover medical expenses.

https://en.wikipedia.org/wiki/Liebeck_v._McDonald%27s_Restau...

If instead this happened in another part of the world instead of the USA, I doubt that McDonalds would have had to pay much if anything in a similar situation.

And the point is that it seems that especially in the USA the companies are very avoidant of ever admitting fault for anything happening to their customers, for fear of lawsuits where they have to pay a lot of money to individual people.

Re: AI didn't delete your database, you did

#96

Post by BadBadJellyBean »

imiric wrote: post #76
BadBadJellyBean wrote: post #61

Earlier quoted context omitted.

Well the user chose the tool. The tool is an LLM. LLMs are non deterministic. You can not predict what comes out ouf an LLM for a given input, especially without weights. This should be known. There is currently no way to prevent this apart from not giving the LLM full control. It will not delete what it can not delete. Use an LLM to write an ansible playbook or some terraform code if you want, but review it, test it…

> There is currently no way to prevent this apart from not giving the LLM full control. It will not delete what it can not delete. But deleting something is just one action you might not want it to take. The recent "agentic" craze is fueled by the narrative pushed by companies and influencers alike that the more access given to an LLM, the more useful it becomes. I think this is ludicrous for the same reasons as you,…

I agree that the marketing is crazy. The dangers are not nearly talked enough about.

Still if I cut off my finger with a bandsaw that is usually my fault. I didn't use tool in a safe way. People have to learn how to use their tools in a safe way. You wouldn't give an intern that much power on day one.

Re: AI didn't delete your database, you did

#98

Post by dylan604 »

stingraycharles wrote: post #35
edot wrote: post #20

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissions would also blame interns for deleting production whatever. Blame should go up, praise should go down. People always invert these.

> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the faul…

> I honestly don’t understand why people blame AI here,

Are you being hyperbolic here? Of course you understand why. Most people would much rather push blame somewhere else, anywhere else, than to accept fault for themselves. Whether that's because of fear of losing job or personal reputation, the reasoning doesn't really matter.

Re: AI didn't delete your database, you did

#99

Post by __mharrison__ »

I think this goes to a broader point: developers aren't necessarily hired to write code.

They're hired to be responsible for some part of the product.

Introducing AI doesn't remove that responsibility.

Folks tend to focus on the code and the tools they're using (maybe I'm cynical from years in the industry). I don't think your boss wants to do your job, even if they could use AI to do it. I think your boss wants to have a headcount, and he wants the headcount to be responsible for the product.

Post reply on HN