Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is not your architect. Stop letting it pretend

hollandtech.net

Post reply on HN
Print view View original

101–110 of 209 posts

Re: Claude is not your architect. Stop letting it pretend

#101

Post by operatingthetan »

jdmichal wrote: post #95
operatingthetan wrote: post #45

Earlier quoted context omitted.

>And it does get people into a lot of trouble. Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?

> Are people just that easily overcome by confident voices? Back in high school, my AP calculus class did some experiments with our teacher's blessing. We'd send a kid out to walk around during class and see how long it took for them to get sent back. Anyway, it ends up that walking around purposely with a piece of paper or envelope, like you're on a mission to deliver it, was a very successful tactic.

I've seen internet comments similar: "put on a yellow vest and carry a clipboard and you can enter any building, anywhere." Confidence is scary, and often misleading.

Re: Claude is not your architect. Stop letting it pretend

#102

Post by wpm »

operatingthetan wrote: post #41
irishcoffee wrote: post #37

Earlier quoted context omitted.

My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.

The apple dictionary says the word means "prepared to obey others unquestioningly." I don't think an inanimate object is capable of "obeying." Or at least that is a very strange way to refer to the act of using a tool.

[dead]

Re: Claude is not your architect. Stop letting it pretend

#103

Post by AndrewKemendo »

DrewADesign wrote: post #88
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

Humans’ general inability to entirely divorce social instincts, responses, and mores while using human language to communicate, especially with something that pantomimes it back, is one of the reasons current chat interfaces are fundamentally flawed. This is working against innate behavior … not something that can be easily switched off. I’ll bet most of the people that can really do it have a hard time intuitively n…

> I’ll bet most of the people that can really do it have a hard time intuitively navigating real social interactions

Bingo. Hi that’s me.

I’ve been trying to teach people how to use LLMs effectively not just dump shit in them but actually talk to them like you would expect a computer to understand and it totally breaks peoples brains

I’m quite successful in helping people get somewhere usable that they weren’t…but to get to the point of fluency with computing systems, and I would argue this is prior to LLMs as well, where you can actually get what you want more reliably out of a computing interaction than you can with a human interaction, is an entirely different way of thinking

That mode of thinking is just generally not accessible to the vast majority of humans. Not because there’s something wrong with them

but it takes somebody who can hold both extremely large scale problems and very very granular specific implementation problems in your head all at once and that is a rare skill.

Re: Claude is not your architect. Stop letting it pretend

#104

Post by JoeAltmaier »

operatingthetan wrote: post #101
jdmichal wrote: post #95

Earlier quoted context omitted.

> Are people just that easily overcome by confident voices? Back in high school, my AP calculus class did some experiments with our teacher's blessing. We'd send a kid out to walk around during class and see how long it took for them to get sent back. Anyway, it ends up that walking around purposely with a piece of paper or envelope, like you're on a mission to deliver it, was a very successful tactic.

I've seen internet comments similar: "put on a yellow vest and carry a clipboard and you can enter any building, anywhere." Confidence is scary, and often misleading.

Or an inspector's hard hat in a construction zone. Nobody wants to confront the inspector.

Re: Claude is not your architect. Stop letting it pretend

#105

Post by kaonwarb »

I assume this post was fully human-written, but ironically, there's something quite LLM-ishly overconfident about this assertion: > They're also confidently wrong about every decision that matters.

Every decision that matters? Some, yes. Is the author only noticing the decisions that go wrong?

Re: Claude is not your architect. Stop letting it pretend

#106

Post by peteforde »

I think it's probably a mistake to make a blanket statement that Claude gets every important thing wrong. It's one of those obviously untrue things that makes a skeptical reader question the validity of the rest of the article.

For what it's worth, Opus tells me that I'm wrong and not to do things all of the time. When I reflect on why that is, it's because of the way that I prompt it. You could say that I am subconsciously avoiding setting both me and the LLM to fail in the way the author projects as inevitable.

Specifically, I don't come to it with prompts that resolve cleanly with "tell me how clever I am" replies. I always present myself as a domain expert - because I am a domain expert - and I make it clear when I am open to getting input on the pros and cons of different paths.

With a conclusion that will be unsurprising to any successful LLM daily drivers, this strategy has been remarkably effective.

Re: Claude is not your architect. Stop letting it pretend

#107

Post by awesome_dude »

saltcured wrote: post #97
operatingthetan wrote: post #45

Earlier quoted context omitted.

>And it does get people into a lot of trouble. Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?

I find it really disturbing, I think because it is illuminating a much more basic problem. It is there in our political and religious histories. We're living through a similar political time right now. A large number of people seem all to ready to find some pervasive authority and subjugate themselves to it. The more concrete machine authority figure is also prevalent in scifi literature. Sometimes, I am not even cer…

Conversely - we tell people who are speaking in public to "Show confidence" - or in job interviews "Hire people who are confident"

We've also pushed back "The more a person knows, the less confident they are" - Dunning Kruger - often used to dismiss over confident people - points out that people are really confident, at first, then that confidence drops away, markedly, but it rebuilds (slowly).

That last rise in confidence is what (I believe) people use as a heuristic on the likely level of knowledge possessed by the speaker (AI or human)

Most engineers know, though, that overconfident people are toxic - the difference between arrogance and genuine confidence in the answer is incredibly difficult to define.

Re: Claude is not your architect. Stop letting it pretend

#108

Post by chongli »

ff317 wrote: post #81
chongli wrote: post #40

Earlier quoted context omitted.

It needs to be subservient It doesn’t. Computer interfaces had no superfluous subservient text for their entire history prior to LLMs. Some of these interfaces have been highly efficient as tools, arguably more efficient than more recent software in many cases. When people complain about LLMs being subservient, they’re not complaining about the tool fulfilling their request. They’re complaining about being forced to…

> Computer interfaces had no superfluous subservient text for their entire history prior to LLMs Clippy would like to help you correct this statement. https://en.wikipedia.org/wiki/Office_Assistant

Not a great example of the way tools need to be, but point well taken. One of the few exceptions that proves the rule and widely despised!

Re: Claude is not your architect. Stop letting it pretend

#110

Post by himata4113 »

I have a good story to share that I came across recently.

Around 2 years ago I had to clean up a mess because someone who doesn't really know what they're doing designed an instancing system for a game. They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed.

Fast forward to today: different company, same person, SAME issues with an AI that is 'allegedly' much better than it was. This time I only heard about these issues and wasn't the one who had to deal with it so I just had a really good laugh.

AI is only as good as the person using it, that's why we have such vast range of what people "claim" AI can do and why everyone has way different opinions of it.

Post reply on HN