Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

61–70 of 329 posts

Re: AI didn't delete your database, you did

#61

Post by BadBadJellyBean »

imiric wrote: post #42

This is missing the point. The issue isn't with the amount of guardrails in place to perform an action. Yes, it is obvious that there should be some in place before doing any critical operation, such as deleting a database. The issue is that the "agent" completely disregarded instructions, which in the age of "skills" and "superpowers" seems like an important issue that should be addressed. Considering that these too…

Well the user chose the tool. The tool is an LLM. LLMs are non deterministic. You can not predict what comes out ouf an LLM for a given input, especially without weights. This should be known.

There is currently no way to prevent this apart from not giving the LLM full control. It will not delete what it can not delete.

Use an LLM to write an ansible playbook or some terraform code if you want, but review it, test it, apply it. Keep backups (3-2-1 rule at minimum).

Letting an LLM have access to everything is just a bad idea and will lead to bad outcomes. You can not replace a person with a mind and experience with an LLM. You can try. But you will probably fail.

Re: AI didn't delete your database, you did

#62

Post by nkassis »

spiderfarmer wrote: post #43
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

I don’t know why people still consider the US the ideal country for starting companies. Everything seems to evolve around taking people to court.

Because it rarely does end up in courts. But having a fair and strong judicial system is a feature not a bug. The parent points out, in the end there must be a way to resolve accountability and ideally it's done in a manner where both parties can be heard and make a case. Find me a better system than a judicial system for this? Mobs?

Re: AI didn't delete your database, you did

#63

Post by biophysboy »

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

[dead]

Re: AI didn't delete your database, you did

#64

Post by lowbloodsugar »

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Humans aren’t any better. That’s why we have OSHA etc. I think you’re hoping for a formal logic based AI and I’ll wager no such thing will ever exist - and if it do, it would try to kill us all.

Re: AI didn't delete your database, you did

#66

Post by onlyrealcuzzo »

2ndorderthought wrote: post #40
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Accountability is the prevailing missing ingredient in us society.

> The problem is that people are now building our world around tooling that eschews accountability.

Management has doing a wonderful job of eschewing accountability for decades.

It's a lot of people's dream to be able to say, yeah, our product doesn't work, but it's not OUR fault, and the client just shrug and grumble ai ai ai, and just put up with it because they know they can't get a better service anywhere else.

It's not MY fault my website is down: it's Amazon's! It's not MY fault my app doesn't work: it's Claude Code's!

Re: AI didn't delete your database, you did

#67

Post by cheschire »

2ndorderthought wrote: post #40
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Accountability is the prevailing missing ingredient in us society.

I don’t think it’s missing, I just think it’s seen as a liability, and American society has been known to absolutely obliterate people who are liable.

Everyone thinks they have the right to judge, and use the massive amounts of available information to do so, even if they haven’t been trained to judge.

Re: AI didn't delete your database, you did

#68

Post by kbrkbr »

pengaru wrote: post #5

wiring up an RNG to your CLI has fairly obvious risks, the root of the problem is ~everyone's treating GenAI as if it's AGI - the rest is popcorn fodder.

That.

"And it confessed in writing" - no, it created probabilistically token after token based on the context without any other access to what happened.

LLMs can't explain themselves in the manner relevant here, much less confess.

Re: AI didn't delete your database, you did

#70

Post by Nicook »

pc86 wrote: post #25
yabones wrote: post #16

"move fast and break things" only sounds good when it's not breaking things in a serious and unfixable way. Maybe we shouldn't take hype mantras as instructive means to an end.

There really shouldn't be any "serious and unfixable way" to break things, especially in a modern company that uses technology in any meaningful way. The fact it's even possible to get into an unrecoverable state is the primary issue.

That's literally always possible? The idea is to put up walls and fail-safes to minimize the chance.
Post reply on HN