"Smoking didn't give you cancer, you did."
AI didn't delete your database, you did
251–260 of 329 posts
Re: AI didn't delete your database, you did
#252The problem with asking an LLM for "its reasoning" after the fact, is that any justification it might give is a post-hoc rationalisation rather than a pre-meditated reason.
Re: AI didn't delete your database, you did
#253Maybe the reason this is so controversial is that people have stopped thinking about "AI" as a bunch of software, just like any other software. If that's you, stop while you still can, you've swallowed a nasty hook and your agency is on the line.
Re: AI didn't delete your database, you did
#254Earlier quoted context omitted.
My team and I are firm that we are the ones accountable. LLMs are a tool like every other. Only that it's non deterministic. But I am the one using the tool. I am the one giving the tool access. I am the one who has to keep everything safe. I have shot myself in the foot using gparted in the past by wiping the wrong disk. gparted wasn't to blame. I was. Letting LLMs work freely without supervision sounds great but it…
> gparted wasn't to blame. I was. These can both be true, especially if/when it has bad defaults. This is why you have things like "type the name of the database you're dropping" safety features - but you also have to name your production database something like "THE REAL DaTabaSe - FIRE ME" so you have to type that and not fall into the trap of ending up with the same name in test/development. AI is particularly sed…
Insufficient - the LLM can figure out what to pass in and pass it in.
I have a production system that I deploy through Claude Code, and initially placed a safeguard like that. About three weeks later it had automated around it.
That’s fine in my case because I’m a professional - I have backups, contingencies in place, etc. If I were non-technical I likely wouldn’t know to do that.
Re: AI didn't delete your database, you did
#255The problem with asking an LLM for "its reasoning" after the fact, is that any justification it might give is a post-hoc rationalisation rather than a pre-meditated reason.
Agreed, and you could get a completely unrelated LLM to generate a similar apology without any of the real context, it would make up reasoning just as effectively.
Re: AI didn't delete your database, you did
#256Don’t put all the eggs into one basket.
It actually helps. I do copy backups to another place as well. One backup is good, but two is better.
Re: AI didn't delete your database, you did
#257Earlier quoted context omitted.
This is kind of the reverse of https://en.wikipedia.org/wiki/Poka-yoke . A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. LLMs .. well, the text interface is uniquely flat. Everything is seemingly as easy as everything else. I worry about the use of humans as sacrificial accountability sinks. The "self-driving car" model already has this: a car which drives its…
> A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder. This is true for almost anything handed to laypeople, but not for a lot of professional tools. Even a plain battery powered drill has very few protections against misuse. A soldering iron has none. Neither do sewing needles; sewing machines barely do, in the sense that you can't stick your fingers in a gap too…
An LLM is a large and complex machine, not a screwdriver. Large and complex [physical] machines are built with safeguards to prevent misuse, injury, etc by regulation.
Re: AI didn't delete your database, you did
#258Earlier quoted context omitted.
I agree that LLMs could be more open about their dangers and that people are bad at judging risks sometimes. Still I think a band saw has very little warning on it and by it's design there is very little anyone can do about me cutting off my finger if I am not careful. LLM companies can do very little about the unpredictability of LLMs. So we have to choose how for we will let it go. In the end the LLM only produces…
A band saw is always a screaming band of bladed death. An LLM is sometimes a buddy, sometimes a mentor, and only sometimes a guy that drops your database.
It's not easy to always remember it's a soulless tool. Sometimes I'm even about to say "thanks" before closing the chat window, until I realize I wouldn't say thanks to my saw or to a random CLI command. But AI, the saw and the random CLI command can all be helpful or destructive. Until the AI shows some signs of consciousness, I'll never treat it as a buddy or a mentor. I'll treat it like an advanced combination of grep, sort and other commands that manipulate text.
It's hard to remember that when it works so amazingly well sometimes. I've been chatting with AI for a few years and every day I'm still amazed and how this is all possible. We've never had this in our lives until a few years ago and now it's changed the way we do a lot of things.
But just like we have to remember the magical machine elves we hallucinate are not really there, we have to constantly remind ourselves that it's an unpredictable soulless tool with many rough edges.
If it helps to treat it like a human, treat it like an idiot savant with autism, schizophrenia, ADHD, psychopathy and a personality disorder who sometimes forgets to take their pills and can start breaking things should a fly lands on their shoulder. You'd listen to them and value their input, but you wouldn't let them in your data center unsupervised as they have no ethics and no honor.
Re: AI didn't delete your database, you did
#259> Automation helps eliminate the silly mistakes that come with manual, repetitive work. We could have easily gone around asking "Why didn't SVN prevent us from deleting trunk?"
Do both? Question bad design and then do whatever you can to work around it.
As an example, that's why flags like this make sense, even if it's a pretty specific use case and there won't be many people using that option at all, preventing stupid default behavior is a good idea: https://superuser.com/a/742735
Re: AI didn't delete your database, you did
#260Earlier quoted context omitted.
> Retain full human responsibility and accountability for any consequences arising from the use of AI systems So if the tool doesn't do what it's supposed to be doing we should blame the user instead of the company that made the tool?
Your comment is a perfect example of not caring about nuance. More charitably, it comes from a place of naivete about how LLMs work. LLMs are non-deterministic [0]. They can't be trusted to fully follow your prompts. As such, you have to be careful about what permissions they have. Like...I use Claude Code. I allow it to run some shell commands that only read (grep, ls, find, etc.). I will never allow it to run Pytho…
> LLMs are non-deterministic
For someone who complains about a lack of nuance it's surprising you're completely missing my point. The lack of trust is precisely my point.
Either AI companies are made accountable for providing un unreliable service or they need to stop selling and marketing these LLMs as if they were infallible.