Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

311–320 of 329 posts

Re: AI didn't delete your database, you did

#311

Post by account42 »

bilbo0s wrote: post #90
onlyrealcuzzo wrote: post #66

Earlier quoted context omitted.

> The problem is that people are now building our world around tooling that eschews accountability. Management has doing a wonderful job of eschewing accountability for decades. It's a lot of people's dream to be able to say, yeah, our product doesn't work, but it's not OUR fault, and the client just shrug and grumble ai ai ai, and just put up with it because they know they can't get a better service anywhere else. I…

Well just to be clear from a legal perspective, in the case of AI, as long as AI is "property", the owners, developers, and/or users will be held liable for things like the hypothetical fatal car accident that Sussman posits. Currently, from a legal perspective, AI is considered a "tool" without legal persona. So you sue the developer, the owner, or the user of the AI. (Just kidding, any lawyer worth his/her salt wil…

This doesn't seem to be how it works in practice. "AI" or not, complex systems are a pretty good shield from accountability in practice today.

Re: AI didn't delete your database, you did

#312

Post by account42 »

QuantumNomad_ wrote: post #95
2ndorderthought wrote: post #75

Earlier quoted context omitted.

List the companies who received a fine worthy of the damage they caused in recent history. List the ones who didn't. It's not about judging. We are socializing the losses to the public and capitalizing the profits for the already wealthy.

We dont know the final amount, as they settled out of court, but in 1992 a woman was awarded hundreds of thousands of dollars by the judge after receiving third degree burns from a coffee at a McDonalds. She had originally asked for $20,000 to cover medical expenses. https://en.wikipedia.org/wiki/Liebeck_v._McDonald%27s_Restau... If instead this happened in another part of the world instead of the USA, I doubt that M…

Yes, McDonals paid one injured party out of many who may - whose injuries were not big enough or who did not have the time/money/energy for a lawsuit but whose combined damages could easily be more than hundreds of thousands.

Re: AI didn't delete your database, you did

#313

Post by philipallstar »

xboxnolifes wrote: post #178
philipallstar wrote: post #85

Earlier quoted context omitted.

> The problem is that people are now building our world around tooling that eschews accountability. If you tell Terraform the wrong thing it will remove your database and not be accountable either.

But nobody would try to excuse their mistake with "terraform deleted my database". Or if a small handful of people did try, every single other person would call them out.

Yes, I agree. And the same should be true for AI tools.

Re: AI didn't delete your database, you did

#314

Post by QuantumNomad_ »

rmunn wrote: post #294
2ndorderthought wrote: post #139

Earlier quoted context omitted.

When healthcare is free the amount of damages is harder to claim maybe?

If healthcare is free then you aren't paying your doctors. What you mean is "when healthcare is paid for by other people ", and in that case the cost of the healthcare is still calculable.

In socialist countries we have understood that it is better for the individual that everyone pays a share of what it costs to maintain a functioning healthcare system, and making it available to everyone for free.

Re: AI didn't delete your database, you did

#315

Post by engeljohnb »

locknitpicker wrote: post #307
engeljohnb wrote: post #184

Earlier quoted context omitted.

>Just because you can claim that a person kick started something Kick started what? If you decided to give an LLM access to your database, it's completely on you when you when it does something you don't want. You should've known better. If all you "kickstart" is an LLM generating text that you can use however you decide, there will never be anything to worry about from the LLM. > Let's put things in perspective: if…

> Kick started what? If you decided to give an LLM access to your database, it's completely on you when you when it does something you don't want. You should've known better. You don't decide anything. You prompt a coding assistant to apply a change to a repository and without intervention it asserts there's a typo in a table name and renames it. The agent validates the change by running tests and integration tests f…

> You don't decide anything. You prompt

Right there. That's where you made the decision, and that's where you went wrong.

>I don't think you fully understand how agents and coding assistants work. By design they are completely autonomous and work by reusing your own personal credentials. As they are completely autonomous, they can apply arbitrary changes.

Yes, and someone somewhere decided to use a coding assistant that can apply arbitrary changes, knowing full well that LLMs are known to hallucinate and make mistakes, and not rarely.

> Why do you even presume that people explicitly grant permissions? That's not how it works at all.

How can you say this with a straight face? Did the LLM hack its way into your workflow? No, someone chose to use it. It doesn't matter that it's autonomous once you enter your prompt. That's actually all the more reason to not allow it to make changes.

> If you wish to criticize a topic, the very least you must do is get acquainted with the topic. Otherwise you'll spend your time arguing with your misplaced beliefs instead if the actual problem.

And if you want to argue with me, you need to actually read and understand what I'm saying.

Say you're staying in the hopsital, and instead of a human nurse making adjustments to your medication, the doctor has an LLM that interfaces directly with the pharmacy and your IV pump. It can make changes to your medication and your dosage without a human ever being involved.

If you overdose because the LLM hallucinated, would you consider an acceptable excuse if the doctor says

"I don't think you fully understand how agents and nursing assistants work. By design they are completely autonomous and work by reusing your own personal credentials. As they are completely autonomous, they can apply arbitrary changes. I mean, nursing assistants nowadays prescribe their own meds on the fly. Why do you even presume that people explicitly grant permissions? That's not how it works at all."

I wouldn't.

Re: AI didn't delete your database, you did

#317

Post by tremon »

techaqua wrote: post #298
rglover wrote: post #234

Earlier quoted context omitted.

Give it a name, but something non-human like "thingbot" or "tacosplosion." "Hey tacosplosion, generate me an exploding taco image."

You already failed. We don't say hey to machines.

Indeed. We address them as Siri, Alexa or Google.

Re: AI didn't delete your database, you did

#318

Post by gitaarik »

edot wrote: post #20

This is why you don’t hire interns! They can delete things and cause havoc! The same people who would blame AI for their failing to properly configure permissions would also blame interns for deleting production whatever. Blame should go up, praise should go down. People always invert these.

You can however let them work in a sandboxed dev environment where if they break things it doesn't affect the productions system.

Re: AI didn't delete your database, you did

#319

Post by [deleted] »

bsoles wrote: post #287
BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> gparted wasn't to blame. I was. I don't know about gparted, but I always felt that "rm -i" should have been the default. The safe option should always be the default and you can optionally make it unsafe. Same goes with "mv -i".

[deleted]

Re: AI didn't delete your database, you did

#320

Post by Uhhrrr »

p91paul wrote: post #291
Uhhrrr wrote: post #244

Earlier quoted context omitted.

If you look at the article, its title is, "An AI agent deleted our production database. It confessed in writing." To me this seems to be pretty clearly focusing on the AI agent. Then if you read the article, it attributes a lot of actions to the agent, and zero responsibility to the humans running the agent. It seems to be an anti-ad for the person's business.

I did not say they did not read the title of the article. They clearly did. It's the rest of the content that was lost, such as the long focus on the sloppiness of the API provider

Looking again at the article, I see that there are about 5 paragraphs about Railway and 30 about the agent (and, again, zero reflection about their own culpability).
Post reply on HN