91–100 of 209 posts
Earlier quoted context omitted.
This is the whole point. “Sorry, the AI said that you are not approved for this cancer treatment, it’s not going to be covered.” “Sorry, the AI said that you were at the scene when the crime took place.” “Sorry, the AI has flagged your account for inappropriate content.” “Sorry, the AI says that you are too risky to lend to.” …
Computer says no, but worse.
I think the article has the correct message, but I disagree with this: > It’s just incapable of the thing that makes a real architect valuable: saying “no.” From my experience Claude is excellent at saying "no". It won't say "no" if the prompt doesn't call for it (it won't say "no" to your direct request to do something, usually). But it offers good critique and happily pushes back if you make it clear that that's a…
Earlier quoted context omitted.
>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.
AI should be subservient in the same way a hammer is subservient.
Earlier quoted context omitted.
The apple dictionary says the word means "prepared to obey others unquestioningly." I don't think an inanimate object is capable of "obeying." Or at least that is a very strange way to refer to the act of using a tool.
You can refer to it however you want, the outcome is the same.
Earlier quoted context omitted.
AI uses a high confidence tone - likely because its training data is heavy on authoritative texts/reference books. And it does get people into a lot of trouble. I have got into trouble with it when it is extremely confident about something I am not very familiar with (as recently as two weeks ago with Claude). I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition…
>And it does get people into a lot of trouble. Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?
Back in high school, my AP calculus class did some experiments with our teacher's blessing. We'd send a kid out to walk around during class and see how long it took for them to get sent back. Anyway, it ends up that walking around purposely with a piece of paper or envelope, like you're on a mission to deliver it, was a very successful tactic.
Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…
Humans’ general inability to entirely divorce social instincts, responses, and mores while using human language to communicate, especially with something that pantomimes it back, is one of the reasons current chat interfaces are fundamentally flawed. This is working against innate behavior … not something that can be easily switched off. I’ll bet most of the people that can really do it have a hard time intuitively n…
Earlier quoted context omitted.
AI uses a high confidence tone - likely because its training data is heavy on authoritative texts/reference books. And it does get people into a lot of trouble. I have got into trouble with it when it is extremely confident about something I am not very familiar with (as recently as two weeks ago with Claude). I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition…
>And it does get people into a lot of trouble. Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?
The more concrete machine authority figure is also prevalent in scifi literature. Sometimes, I am not even certain if the author is doing this to examine this issue versus just leaning into it as either appealing to themselves or to the perceived audience.
For fun I've been vibe coding something I know well: toolchains. Maybe not the right thing to vibe code. But I can more or less judge the quality of the output. When left to its own devices with the instructions "make an assembler for the architecture in ISA.md" -- well Claude picked Python as the implementation language. Tokens lifted through a bunch of regex. No expression parser! Oh dear. My first assembler was li…
LLMs are bringing us back to all the “proper” software engineering stuff that we’ve always known we should be doing, but until now we never had enough time/people/money to do it right. Brainstorming and research before writing a design. Writing a design or spec before writing the code. Comprehensive unit tests. Etc etc etc. Like you, I get vastly better output from the tool when I create a detailed spec in markdown b…
Earlier quoted context omitted.
My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.
You’re still anthropomorphizing. They’re not communicating, you’re just being observant.
Since we are talking about hammers: you hit the nail on the head.
The only consciousness, observing, and thinking happening when a person is using an LLM is happening in the person's brain. We project our own consciousness onto them, and that is the anthropomorphizing part. Essentially we empathize with the object because they are designed to respond like a person. The "conversation" is purely an illusion.
Earlier quoted context omitted.
>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.
AI should be subservient in the same way a hammer is subservient.
A hammer isn’t subservient, it doesn’t have the capacity to be. Saying a hammer is subservient is stretching the definition for literary flourish, but it doesn’t actually make a lot of sense.
The definition that came up for subservient when I checked was “prepared to obey others unquestioningly“.