Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is not your architect. Stop letting it pretend

hollandtech.net

Post reply on HN
Print view View original

81–90 of 209 posts

Re: Claude is not your architect. Stop letting it pretend

#81

Post by ff317 »

chongli wrote: post #40
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

It needs to be subservient It doesn’t. Computer interfaces had no superfluous subservient text for their entire history prior to LLMs. Some of these interfaces have been highly efficient as tools, arguably more efficient than more recent software in many cases. When people complain about LLMs being subservient, they’re not complaining about the tool fulfilling their request. They’re complaining about being forced to…

> Computer interfaces had no superfluous subservient text for their entire history prior to LLMs

Clippy would like to help you correct this statement.

https://en.wikipedia.org/wiki/Office_Assistant

Re: Claude is not your architect. Stop letting it pretend

#82

Post by ambicapter »

operatingthetan wrote: post #41
irishcoffee wrote: post #37

Earlier quoted context omitted.

My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.

The apple dictionary says the word means "prepared to obey others unquestioningly." I don't think an inanimate object is capable of "obeying." Or at least that is a very strange way to refer to the act of using a tool.

You can refer to it however you want, the outcome is the same.

Re: Claude is not your architect. Stop letting it pretend

#83

Post by xivzgrev »

This gets at the biggest gap I see in AI discussions - the accountability

It doesn't disappear when you make 1 person do the work of 3. It simply is aggregated

Suppose you had a pod of a PM, designer, and analyst. Leadership lays off the designer and analyst and now the PM can move faster with AI. Hooray!

Well...when the complaints about how it looks on xyz device roll in, who is implementing that? Or you launch the product with much fan fare, adoption is terrible - you double check the numbers and oops, the sizing you had from Claude was actually 10x off.

Who is holding the bag? You are! Not Claude

I'm convinced this is one reason we are seeing slower than expected adoption of AI broadly in tech companies, because it's hard to trust - we know Claude can make mistakes but how do you know what's right vs not? Most people don't want to sense check so they just keep doing the work the way they know best.

I think this could be one thing that pops the AI bubble - execs try to force this, people go along, and results are not any better for this reason. Sure you save some salaries and ship more quickly, but you don't build the right thing and you are fixing more things after launch. Which one is actually better?

Re: Claude is not your architect. Stop letting it pretend

#84

Post by aksss »

devin wrote: post #52
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction. One reaction to this might be "well that's not what I mean, that suggests you're prompting with too much directionality" which could further be con…

A good habit to build is knowing when to abandon a session and start over rather than trying to correct. There’s room for correction but you can kind of smell when the whole discussion has become rotten and inefficient. Sometimes it’s just better to use the session as rubber ducking to learn how to correctly articulate what you’re after and start a new session with that clean and correctly articulated foundation.

Re: Claude is not your architect. Stop letting it pretend

#85

Post by gobdovan »

amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

> AI is a tool. It needs to be subservient

Fun experiment, chat with an LLM and swap roles. Tell it you're gonna be the assistant and them the assisted. I found they're pretty bad at using a human for what they're good for.

Re: Claude is not your architect. Stop letting it pretend

#86

Post by dyauspitr »

retrac wrote: post #2

For fun I've been vibe coding something I know well: toolchains. Maybe not the right thing to vibe code. But I can more or less judge the quality of the output. When left to its own devices with the instructions "make an assembler for the architecture in ISA.md" -- well Claude picked Python as the implementation language. Tokens lifted through a bunch of regex. No expression parser! Oh dear. My first assembler was li…

It doesn’t even to be that complex. You can just say do comprehensive research and analysis in the space and give me an implementation plan. Then if it is 20 steps, I ask it to implement 3-5 at once. It’s essentially been one shot for everything I can throw at it.

Re: Claude is not your architect. Stop letting it pretend

#87

Post by jstummbillig »

devin wrote: post #52
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction. One reaction to this might be "well that's not what I mean, that suggests you're prompting with too much directionality" which could further be con…

> The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction.

I don't think that is the flip side. That's just obviously bad. Everything that is obviously bad, the model makers will also ~notice and work to make better. They seem to be a competent and attentive bunch, on the whole.

Re: Claude is not your architect. Stop letting it pretend

#88

Post by DrewADesign »

amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

Humans’ general inability to entirely divorce social instincts, responses, and mores while using human language to communicate, especially with something that pantomimes it back, is one of the reasons current chat interfaces are fundamentally flawed. This is working against innate behavior… not something that can be easily switched off. I’ll bet most of the people that can really do it have a hard time intuitively navigating real social interactions.

It also makes it an incredible tool for manipulation.

Re: Claude is not your architect. Stop letting it pretend

#89

Post by throwatdem12311 »

operatingthetan wrote: post #29
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

AI should be subservient in the same way a hammer is subservient.

Re: Claude is not your architect. Stop letting it pretend

#90

Post by airstrike »

awesome_dude wrote: post #39
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

AI uses a high confidence tone - likely because its training data is heavy on authoritative texts/reference books. And it does get people into a lot of trouble. I have got into trouble with it when it is extremely confident about something I am not very familiar with (as recently as two weeks ago with Claude). I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition…

> I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition, and it has steadfastly refused to take my point (last week)

Ironically, trying to argue with Claude about the limitations of LLMs and AI in general today is quite hard. It refuses to yield, likely due to Anthropic tweaking it aggressively

Post reply on HN