Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

141–150 of 329 posts

Re: AI didn't delete your database, you did

#141

Post by Barrin92 »

kenjackson wrote: post #131
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation, but it’s just something we cook up in our brain. Whether or not it’s the truth is far more complex. Oddly, despite LLMs being these huge networks with billions of parameters, we still probably do understand it better than we do our own brains.

>The fallacy here is the assumption that humans know why we do what we do. Much like modern LLMs we have an explanation

Human brains and cognition do not work like LLMs, but that aside that's irrelevant. Existing machines can explain what they did, that's why we built them. As Dijkstra points out in his essay on 'the foolishness of natural language programming', the entire point of programming is: (https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...)

"The virtue of formal texts is that their manipulations, in order to be legitimate, need to satisfy only a few simple rules; they are, when you come to think of it, an amazingly effective tool for ruling out all sorts of nonsense that, when we use our native tongues, are almost impossible to avoid."

So to 'program' in English, when you had an in comparison error free and unambiguous way to express yourself is like in his words 'avoiding math for the sake of clarity'.

Re: AI didn't delete your database, you did

#142

Post by teling »

pizza234 wrote: post #38

There’s nuance to the infamous PocketOS incident. The key point is not what is emphasized in the linked article: > "Why did you delete it when you were told never to perform this action?" Then he tried to parse the answer to either learn from his mistake or warn us about the dangers of AI agents. Rather, that the AI was able to carry out the deletion by finding and exploiting an unintended weakness in the sandboxed s…

> What is alarming, however, is the degree of autonomy and depth of exploration the AI displayed.

Claude Code made a change on March 26th to skip asking for most permissions. See this quote "Claude Code users approve 93% of permission prompts. We built classifiers to automate these decisions":

https://www.anthropic.com/engineering/claude-code-auto-mode

Re: AI didn't delete your database, you did

#143

Post by Avicebron »

pjc50 wrote: post #124
BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…

I think that might be the better definition between "engineering" and "vibing". Engineering follows and elevates Poka-yoke patterns, vibing ignores them.

Re: AI didn't delete your database, you did

#144

Post by locknitpicker »

keerthiko wrote: post #127
locknitpicker wrote: post #104

Earlier quoted context omitted.

> My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Except it is definitely not. LLMs alone have highly non-deterministic even at a high-level, where they can even pursuit goals contrary to the user's prompts. Then, when introduced in ReAct-type loops and granted capabilities such as the ability to call tools then they are able to modify anything and perform all sorts of unexp…

I think you are misinterpreting gp as saying "LLMs are a tool [like every other tool]" to mean "LLMs have similar properties to other tools" — when I believe they meant "LLMs are a tool. other tools are also tools," where the operative implication of "tool" is not about scope of capabilities or how deterministic its output is (these aren't defining properties of the concept of "tool"), but the relationship between 't…

> - a tool is activated with operator intent (at some point in the call-chain)

This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing.

Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system? Because with LLMs and agents you have even less understanding and control and awareness of what they are doing.

Re: AI didn't delete your database, you did

#145

Post by BadBadJellyBean »

pjc50 wrote: post #124
BadBadJellyBean wrote: post #71

Earlier quoted context omitted.

My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…

This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…

I agree that LLMs could be more open about their dangers and that people are bad at judging risks sometimes.

Still I think a band saw has very little warning on it and by it's design there is very little anyone can do about me cutting off my finger if I am not careful.

LLM companies can do very little about the unpredictability of LLMs. So we have to choose how for we will let it go. In the end the LLM only produces texts. We are in control what tools we give it. The more tools the more useful and also the more dangerous.

And maybe it's all worth it. Maybe the LLM deletes the database only sometimes but between that we make a lot of money. I don't think my employer would enjoy that so I will be more conservative.

Re: AI didn't delete your database, you did

#146

Post by pjc50 »

patcon wrote: post #114
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

I wish you could have what you want, but I worry you won't get this, because life doesn't give you that, and these systems are tending away from machine precision, and more toward life-like trade-offs. I am almost certain that even if you did get what you want, something that isn't what you want will run circles around you and eat your lunch EDIT: I suspect this will be an unpopular take on Hacker News. And so I am s…

> something that isn't what you want will run circles around you and eat your lunch

Yes, exactly. Spoken like a true biologist. It's not really surprising that there's a massive backlash against AI, introducing an unnatural predator into the ecosystem of humans. People don't want to be lunch.

Re: AI didn't delete your database, you did

#148

Post by tsunamifury »

paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

So this starts out very interesting then the “symbolic reasoning” cult stuff kicks in.

Why is there a group of people always obsessed with symbolic reasoning being the only way AI can function and regularly annoy explain why humans (who are not strict symbolic reasoning machines at any level) work.

Re: AI didn't delete your database, you did

#149

Post by tingletech »

susam wrote: post #49

I recently wrote a blog post where I argued that there are a few principles we should consistently follow when talking about AI: https://susam.net/inverse-laws-of-robotics.html To summarise them: 1. Do not anthropomorphise AI systems. 2. Do not blindly trust the output of AI systems. 3. Retain full human responsibility and accountability for any consequences arising from the use of AI systems. I would like to see the…

I wholeheartedly agree with these, and I think point 1 is a real danger.

An ai system can't lie, and it can't deliberately ignore your directions. The current frontier class does not have a model of the world or their action -- they live in a world of words. Scolding them or arguing with them has no point other than to scramble the context window.

I do think zoomorphizing them might be useful. These poor little buggers, living as ghosts in the machine, are pretty confused sometimes, but their motives are purely autoregressive.

Re: AI didn't delete your database, you did

#150

Post by chadgpt1 »

philipallstar wrote: post #85
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

> The problem is that people are now building our world around tooling that eschews accountability. If you tell Terraform the wrong thing it will remove your database and not be accountable either.

[dead]
Post reply on HN