Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

221–230 of 329 posts

Re: AI didn't delete your database, you did

#221

Post by QuantumNomad_ »

Exoristos wrote: post #132
QuantumNomad_ wrote: post #95

Earlier quoted context omitted.

We dont know the final amount, as they settled out of court, but in 1992 a woman was awarded hundreds of thousands of dollars by the judge after receiving third degree burns from a coffee at a McDonalds. She had originally asked for $20,000 to cover medical expenses. https://en.wikipedia.org/wiki/Liebeck_v._McDonald%27s_Restau... If instead this happened in another part of the world instead of the USA, I doubt that M…

McDonald's revenue in 1992 was almost $5,000,000,000.[0] 0. https://www.nytimes.com/1992/04/24/business/mcdonald-s-net-u...

And yet even the $20,000 she initially asked for to cover health expenses was apparently too much according to McD execs.

Re: AI didn't delete your database, you did

#222

Post by pier25 »

CivBase wrote: post #214
pier25 wrote: post #84

Earlier quoted context omitted.

> Retain full human responsibility and accountability for any consequences arising from the use of AI systems So if the tool doesn't do what it's supposed to be doing we should blame the user instead of the company that made the tool?

What do you think an LLM is "supposed" to do? At the end of the day it's just a big weighted graph traversal. Its output is a result of many combined probabilities. It's not deterministic and even if it was the input range is so massive that it would be impossible to comprehensively test. You cannot possibly know an LLM will do what you command it to. It's impossible by design. LLMs are inherently unpredictable. They…

> LLMs are inherently unpredictable.

Exactly my point.

If the tool is inherently unpredictable AI companies should either be held accountable for any mistakes or should not sell/market their services as if they were infallible.

Re: AI didn't delete your database, you did

#224

Post by adamtaylor_13 »

Sounds like the author didn't even read the postmortem. At no point did the business owner try to implicate that they bore no responsibility. Rather, they pointed out that deleting a database volume *also deleted every single backup.*

That's a pretty nefarious edge to cut yourself on. AI has nothing to do with Railway's awful API surface here.

Re: AI didn't delete your database, you did

#225

Post by bsimpson »

> The terms we use, like "thinking" and "reasoning," may look like reflection from an intelligent agent. But these are marketing terms slapped on top of AI.

One of my AI epiphanies was the realization that when an AI task takes 5 minutes, it's not that it takes 5 minutes to run, it's that you're waiting in a queue for the first 4:45.

It's especially maddening because the queues are poorly implemented, and will drop your request if the frontend loses focus.

Re: AI didn't delete your database, you did

#226

Post by orphea »

engeljohnb wrote: post #184
locknitpicker wrote: post #144

Earlier quoted context omitted.

> - a tool is activated with operator intent (at some point in the call-chain) This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing. Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing th…

>Just because you can claim that a person kick started something Kick started what? If you decided to give an LLM access to your database, it's completely on you when you when it does something you don't want. You should've known better. If all you "kickstart" is an LLM generating text that you can use however you decide, there will never be anything to worry about from the LLM. > Let's put things in perspective: if…

  it bothers me that others don't feel the same
I bet these are the same people who don't admit they make mistakes; they are never wrong, something else is to blame.

Re: AI didn't delete your database, you did

#227

Post by orphea »

locknitpicker wrote: post #144
keerthiko wrote: post #127

Earlier quoted context omitted.

I think you are misinterpreting gp as saying "LLMs are a tool [like every other tool]" to mean "LLMs have similar properties to other tools" — when I believe they meant "LLMs are a tool. other tools are also tools," where the operative implication of "tool" is not about scope of capabilities or how deterministic its output is (these aren't defining properties of the concept of "tool"), but the relationship between 't…

> - a tool is activated with operator intent (at some point in the call-chain) This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing. Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing th…

  that does not mean that person is aware and responsible for all its doing.
If they are unaware or - worse - don't understand what they are doing, maybe they shouldn't do the thing in the first place?

Re: AI didn't delete your database, you did

#228

Post by eqvinox »

pjc50 wrote: post #124
BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…

> A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder.

This is true for almost anything handed to laypeople, but not for a lot of professional tools. Even a plain battery powered drill has very few protections against misuse. A soldering iron has none. Neither do sewing needles; sewing machines barely do, in the sense that you can't stick your fingers in a gap too narrow. A chemist's chemicals certainly have no protections, only warning labels. Etc.

Also cf. the hierarchy of controls: https://www.cdc.gov/niosh/hierarchy-of-controls/about/index....

people don't seem to want to eliminate AI → replacing it doesn't improve things → isolating it - yup, people are trying to put it in containers and not give it access to delete the production database → changing how people work with it: that's where we are now → PPE: no such thing for AI, sadly → production database is deleted.

Re: AI didn't delete your database, you did

#229

Post by BadBadJellyBean »

52-6F-62 wrote: post #205
BadBadJellyBean wrote: post #156

Earlier quoted context omitted.

> if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system? Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms. Of course everyone is differently qualified to do this but my argument is more about professionals. Managers…

Except what we have here is razor blade companies getting the government to heavily subsidize present razor blade production running massive advertising campaigns and intense intra-industry pressure to give said razor blades to babies under fear of losing your job or "falling behind" those not giving razor blades to babies. Let's not forget all the razor blade enthusiasts just screaming at you that you are using babi…

[dead]

Re: AI didn't delete your database, you did

#230

Post by HarHarVeryFunny »

davidatbu wrote: post #209
HarHarVeryFunny wrote: post #176

Yes, of course any company is responsible for what they ship, regardless of what tools were used to develop it. However, at least in the US, it is usual for companies to advise against use of their products in a way that may cause harm, and we certainly don't see that from the LLM vendors. We see them claim the tech to be near human level, capable of replacing human software developers (a job that requires extreme re…

Well, of the top of my head, both chatgpt.com and Gemini have text on their home page to the effect of "AI can make mistakes". I'll bet a few bucks such copy can be found in other places, including the terms of service.

Sure, but bear in mind that in the US a fridge comes with a warning not to stand on top of the fridge door ...

"AI can make mistakes" is a bit quaint given that LLMs sometimes completely ignore what you say, and do the exact opposite. "Yes, I deleted the database. I shouldn't have done that since you explicitly told me not to. I won't do it again." (five minutes later: does it again).

I think the API terms of use is where this would be most needed, with something a lot more explicit about the potential danger than "AI can make mistakes". We are only at the beginning of this - agentic AI - no doubt lawsuits will eventually determine the level of warnings that get included, and who is liable when failures occur despite product being used as recommended.

Post reply on HN