Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

151–160 of 329 posts

Re: AI didn't delete your database, you did

#151

Post by jmalicki »

PessimalDecimal wrote: post #111
jmalicki wrote: post #102

Earlier quoted context omitted.

Formal logic AI systems have existed and were popular in the 1980s. One of the problems is that they don't work - in the real world there are no firm facts, everything is squishy, and when you try to build a large system you end up making tons of exceptions for special cases until it becomes completely untenable. Non-deterministic systems that work probabilistically are just superior in function to that, even if it m…

I don't know what definition of AI you're using, but plenty of ML algorithms operate deterministically, let alone most other logic programmed into a computer. I don't see how your statement can be right given that these other software systems also operate in the real world.

ML run a GPU that uses matrix multiplies isn't deterministic unless you go through great pains to lock things down at the expense of performance.

Re: AI didn't delete your database, you did

#152

Post by chadgpt1 »

paroneayea wrote: post #57
spiderfarmer wrote: post #43

Earlier quoted context omitted.

I don’t know why people still consider the US the ideal country for starting companies. Everything seems to evolve around taking people to court.

The point is not primarily the court. The court is an example of someplace where we have accountability, but we build accountability mechanisms as foundational to most of our computing. Tracebacks, debuggers, logging, etc. We put enormous resources into not only the bad case, but the potential that a bad case could occur. When something goes wrong, we want to know why, and we want to make sure that something bad like…

[dead]

Re: AI didn't delete your database, you did

#153

Post by zahlman »

amluto wrote: post #51
stingraycharles wrote: post #35

Earlier quoted context omitted.

> This is why you don’t hire interns! I’d like to rephrase this as: this is why you don’t give interns permissions to delete your prod database. This is a process failure, not an AI failure. I honestly don’t understand why people blame AI here, when you literally gave AI permissions to do exactly this. It’s like blaming AWS for exposing some database to the public. That’s just not AWS’ fault. Neither is this the faul…

There is a major issue with current AI tools that they want to effectively grant access to everything their user has access to. The whole sandbox structure is wrong (although various people have vibe coded assorted improvements).

If you want them to be able to write code and then run tests on that code, it can be a bit difficult to restrict access meaningfully....

Re: AI didn't delete your database, you did

#154

Post by zahlman »

susam wrote: post #49

I recently wrote a blog post where I argued that there are a few principles we should consistently follow when talking about AI: https://susam.net/inverse-laws-of-robotics.html To summarise them: 1. Do not anthropomorphise AI systems. 2. Do not blindly trust the output of AI systems. 3. Retain full human responsibility and accountability for any consequences arising from the use of AI systems. I would like to see the…

>1. Do not anthropomorphise AI systems.

This is maddeningly difficult IMX.

Re: AI didn't delete your database, you did

#155

Post by mystraline »

BadBadJellyBean wrote: post #71
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

> LLMs are a tool like every other. Only that it's non deterministic.

If you stay away from the corporate SaaS token vendors, and run your own, you will find LLMs are deterministic, purely based on the exact phrase on input. And as long as the context window's tokens are the same, you will get the same output.

The corporate vendors do tricks and swap models and play with inherent contexts from other chats. It makes one-shot questions annoying cause unrelated chats will creep into your context window.

Re: AI didn't delete your database, you did

#156

Post by BadBadJellyBean »

locknitpicker wrote: post #144
keerthiko wrote: post #127

Earlier quoted context omitted.

I think you are misinterpreting gp as saying "LLMs are a tool [like every other tool]" to mean "LLMs have similar properties to other tools" — when I believe they meant "LLMs are a tool. other tools are also tools," where the operative implication of "tool" is not about scope of capabilities or how deterministic its output is (these aren't defining properties of the concept of "tool"), but the relationship between 't…

> - a tool is activated with operator intent (at some point in the call-chain) This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing. Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing th…

> if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system?

Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms.

Of course everyone is differently qualified to do this but my argument is more about professionals. Managers should know better than to blindly trust LLM companies. Engineers should take better care what they allow LLMs to do and what tools they give them.

There is a difference between "I couldn't have known" and "I didn't know". You can know that LLMs are not trustworthy. You couldn't have know what they do but you already knew that trusting them blindly might be bad.

You could know that giving a baby a razor blade is a bad idea. You can't know what exactly will happen but you might have a pretty good idea that it will probably be not good.

Re: AI didn't delete your database, you did

#157

Post by armada651 »

kbrkbr wrote: post #78
imiric wrote: post #42

This is missing the point. The issue isn't with the amount of guardrails in place to perform an action. Yes, it is obvious that there should be some in place before doing any critical operation, such as deleting a database. The issue is that the "agent" completely disregarded instructions, which in the age of "skills" and "superpowers" seems like an important issue that should be addressed. Considering that these too…

An LLM generates plausible text token by token. It is at its core a deterministic function with some randomization and some clever tricks to make it look like an agent dialoguing or reasoning. Plausible text sometimes is right, sometimes not. Humans have a world model, a model of what happens. LLMs have a model of what humans would plausibly say. The only good guardrail seems human-in-the-loop.

This is such a motte-and-bailey argument. Whenever people point out LLMs aren't actually intelligent then you're an anti-AI Luddite. But whenever an AI does something catastrophically dumb it's absolved of all responsibility because "it's just predicting the next token".

I'm getting so tired of this.

Re: AI didn't delete your database, you did

#158

Post by QuantumNomad_ »

pjc50 wrote: post #134
QuantumNomad_ wrote: post #95

Earlier quoted context omitted.

We dont know the final amount, as they settled out of court, but in 1992 a woman was awarded hundreds of thousands of dollars by the judge after receiving third degree burns from a coffee at a McDonalds. She had originally asked for $20,000 to cover medical expenses. https://en.wikipedia.org/wiki/Liebeck_v._McDonald%27s_Restau... If instead this happened in another part of the world instead of the USA, I doubt that M…

This is such a litmus test, this case. Yes, America does weird things with punitive damages. But the injuries were really severe and the negligence significant. More often you get class action lawsuits where everyone involved gets mailed a cheque for $3. It's not just America. McDonald's UK got involved in the UK's biggest ever libel case. https://en.wikipedia.org/wiki/McLibel_case ; leaflets distributed in 1985 ende…

> https://en.wikipedia.org/wiki/McLibel_case

Seems kind of an opposite situation. There it was McDonalds suing a pair of people, not the other way around. And the human rights violation was by the UK government and not McD.

Post reply on HN