Live data from Hacker News

Claude is not your architect. Stop letting it pretend

hollandtech.net

41–50 of 209 posts

Re: Claude is not your architect. Stop letting it pretend

#41

Earlier quoted context omitted.

>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.

The apple dictionary says the word means "prepared to obey others unquestioningly."

I don't think an inanimate object is capable of "obeying." Or at least that is a very strange way to refer to the act of using a tool.

Re: Claude is not your architect. Stop letting it pretend

#42
post #23

Earlier quoted context omitted.

> It is entirely possible to develop strong systems outside of your current skill and knowledge with methods like this. If this is true how can you confidently make this assertion. You yourself are not in a position to evaluate it, you are just running it through a couple times hoping for a "oh wait, you're right to call me out on that, that is not correct at all".

1. Tell it to find docs and research best practices. 2. Ask for references and read them. > When done properly your own knowledge should have grown to meet the product you end up with.

I've found relying on my own research first for a local LLM works much better. Asking a biased source to find it's own research will result in biased research.

Re: Claude is not your architect. Stop letting it pretend

#43

> "I’m not saying don’t use AI agents. I use Claude Code every day." Irony is using Claude to write a beautifully structured, 2,000-word essay warning the industry about the dangers of letting Claude design things. It’s self-awareness by proxy.

This should be the first comment. I wrote some criticism, mostly because many internal contradictions in the article. Then, I notice the structure...

"The accountability gap" Here’s the question nobody’s asking: when it goes wrong, who carries the bag? (..)

"What to do instead"

"The craft still matters"

Re: Claude is not your architect. Stop letting it pretend

#44
Accountability is the biggest unaddressed challenge for AI implementation.

When one person is able to do too much too quickly, they can create more liability than they can accommodate if something fails.

It is essential that a human is responsible for the utilization of any AI output in the real world, but that is not enough. For our own sakes, we must find ways to minimize the tech-debt bankruptcy blast-radius of those who would utilize (knowingly or unknowingly) AI to create flawed systems upon which others rely.

An example: Jim vibe-codes an extremely popular micropayments app. He hires a few people and sees the company as the WhatsApp of money -- a few engineers and some agentic support staff. It pulls in a few million in VC money -- enough to draw in tens of millions of users. One day, a flaw in the infrastructure causes all of the users' unsalted banking information to be released.

Agentic AI allows that entire list of customers to be exploited rapidly, so the losses for society are in the tens of billions. Jim's company is immediately bankrupt, of course, but there are only a few million dollars to go around.

Today, most of Jim's incentives are to go ahead and build that app. The same is true for his few employees and a small VC contribution. There's not much capital at risk compared with the societal exposure.

How do we ensure that AI users are accountable not just for their actions, but for the size of the risk-exposure that they create?

Re: Claude is not your architect. Stop letting it pretend

#45
post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

AI uses a high confidence tone - likely because its training data is heavy on authoritative texts/reference books. And it does get people into a lot of trouble. I have got into trouble with it when it is extremely confident about something I am not very familiar with (as recently as two weeks ago with Claude). I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition…

>And it does get people into a lot of trouble.

Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?

Re: Claude is not your architect. Stop letting it pretend

#46

I think the article has the correct message, but I disagree with this: > It’s just incapable of the thing that makes a real architect valuable: saying “no.” From my experience Claude is excellent at saying "no". It won't say "no" if the prompt doesn't call for it (it won't say "no" to your direct request to do something, usually). But it offers good critique and happily pushes back if you make it clear that that's a…

Yeah, just read the first couple of paragraphs and then stopped because that’s not my experience at all with Claude Opus 4.6 and 4.7.

If you ask it with a prompt that leaves room for criticism it’ll definitely go for it when warranted.

Re: Claude is not your architect. Stop letting it pretend

#47

If there was ever a "magic prompt" this one comes close: Brainstorm N ways to do X. Sort by probability. Rather than your AI giving you the average response, it tends to sample wider from the input space. Then I can decide which one to go with (or choose something else). Don't outsource all of your thinking.

I've found this surprisingly effective. Higher "thinking levels" may result in more than one approach being considered, but you can also tell your LLM to do brainstorming explicitly: https://photostructure.com/coding/claude-code-replan/

Re: Claude is not your architect. Stop letting it pretend

#48

I think the article has the correct message, but I disagree with this: > It’s just incapable of the thing that makes a real architect valuable: saying “no.” From my experience Claude is excellent at saying "no". It won't say "no" if the prompt doesn't call for it (it won't say "no" to your direct request to do something, usually). But it offers good critique and happily pushes back if you make it clear that that's a…

[deleted]

Re: Claude is not your architect. Stop letting it pretend

#49
As I keep saying, the problem isn't the tools - it's the humans who don't know what they don't know ----- and assume that what they don't know is insignificant ----- and just plow forward with their authority and/or money.

We can describe this without talking about technology - so pre-AI.

Imagine the owner of a construction company firing all the architects. After all, he's been the owner for 15 years. He has led the construction of dozens of projects. He's also rich, and being rich seems to be an ego-multiplier.

Why should he waste money on architects? Or more importantly, why should he allow them to constantly annoy him with pushbacks: "This could be a problem if the sustained wind is greater than ... ".

Those engineers obviously don't know the real world. Their elitist education has made them afraid to make bold decisions. Regulations are anti-progress!

Thankfully, that owner now has AI tools. He doesn't need those not-always-yes-people. He now has a perpetual yes-bot.

So where are we now? We're in the same place we always have been. People need to have the humility to recognize that despite their authority, influence, or wealth, they still need other people. And especially, they need other people to challenge their orders or their requests.

But I don't really see this situation self-correcting. There's now so much money concentrated amongst a few who will spray it over exactly the kind of people who do not want to listen to others that most activity in the future will be for naught. Yes, some unicorns will be fabricated, and some people will make a lot of money; but real value will not be created often.

Therefore, I implore the actual thoughtful creators: Do build things, but do not sell out. Look to the past. Create companies where every employee was valued, and every employee had some voice. Yes, use AI. But test and measure where it really helps. And be skeptical, just as you would if someone came to your door promising a black box that would double your profits.

Post reply on HN