Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is not your architect. Stop letting it pretend

hollandtech.net

Post reply on HN
Print view View original

51–60 of 209 posts

Re: Claude is not your architect. Stop letting it pretend

#51

Post by mlsu »

ISL wrote: post #44

Accountability is the biggest unaddressed challenge for AI implementation. When one person is able to do too much too quickly, they can create more liability than they can accommodate if something fails. It is essential that a human is responsible for the utilization of any AI output in the real world, but that is not enough. For our own sakes, we must find ways to minimize the tech-debt bankruptcy blast-radius of th…

This is the whole point.

“Sorry, the AI said that you are not approved for this cancer treatment, it’s not going to be covered.”

“Sorry, the AI said that you were at the scene when the crime took place.”

“Sorry, the AI has flagged your account for inappropriate content.”

“Sorry, the AI says that you are too risky to lend to.”

…

Re: Claude is not your architect. Stop letting it pretend

#52

Post by devin »

amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction.

One reaction to this might be "well that's not what I mean, that suggests you're prompting with too much directionality" which could further be condensed to "you're prompting wrong". The trouble with this is that _even when I am trying to be extremely precise and avoid biasing the result_, I still will see the output and go "ah shit, I can see it 'aligning' with whatever dumb thing I've just said as if it is a good/plausible direction".

At that point it starts to feel like the prompt is more dice roll than skill at times, which makes me feel like I'm operating a fancy knowledge slot machine.

Re: Claude is not your architect. Stop letting it pretend

#53

Post by Forgeties79 »

ISL wrote: post #44

Accountability is the biggest unaddressed challenge for AI implementation. When one person is able to do too much too quickly, they can create more liability than they can accommodate if something fails. It is essential that a human is responsible for the utilization of any AI output in the real world, but that is not enough. For our own sakes, we must find ways to minimize the tech-debt bankruptcy blast-radius of th…

I have had multiple conversations on HN with people who fight tooth and nail, I mean really ready to die on their hill, because they believe they shouldn’t even have to vet what comes out of an LLM. It’s absolutely baffling to me. The most bizarre excuse is “it codes better than people,” which is not even remotely a given and needs a lot of qualifiers.

I understand there is a push/pull with regards to how much we should let them do, but to not even look at the results before you make them somebody else’s problem? It’s just selfish. There’s no other word for it. You are simply taking the work you were supposed to do it and dumping it on somebody else. These are probably the same people who get upset (rightfully so!) when somebody doesn’t proofread their article/blog before publishing it online.

Everybody wants to use LLM’s to cut corners on their work but nobody wants to be downstream of it. That simply doesn’t work.

Re: Claude is not your architect. Stop letting it pretend

#54

Post by wild_egg »

operatingthetan wrote: post #29
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

We train dogs to be subservient but that doesn't automatically mean we anthropomorphize them

Re: Claude is not your architect. Stop letting it pretend

#55

Post by throwawaysoxjje »

irishcoffee wrote: post #37
operatingthetan wrote: post #29

Earlier quoted context omitted.

>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.

You’re still anthropomorphizing.

They’re not communicating, you’re just being observant.

Re: Claude is not your architect. Stop letting it pretend

#56

Post by [deleted] »

wild_egg wrote: post #54
operatingthetan wrote: post #29

Earlier quoted context omitted.

>anthropomorphism problem. AI is a tool. It needs to be subservient. Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

We train dogs to be subservient but that doesn't automatically mean we anthropomorphize them

[deleted]

Re: Claude is not your architect. Stop letting it pretend

#57

Post by regularfry »

bluegatty wrote: post #14
retrac wrote: post #2

For fun I've been vibe coding something I know well: toolchains. Maybe not the right thing to vibe code. But I can more or less judge the quality of the output. When left to its own devices with the instructions "make an assembler for the architecture in ISA.md" -- well Claude picked Python as the implementation language. Tokens lifted through a bunch of regex. No expression parser! Oh dear. My first assembler was li…

So where AI has deterministic inputs and outputs it is extremely good to the point I think that there's a theoretical issue around computational there. Like - it can do the work for us. It jives with post training and verifiable rewards. The reason AI doesn't do well at 'architecture' is 1) are are bad at it and have given it a lot of mush and 2) we don't have good abstractions for it. The result is - you stick to 'v…

I have found that if you give it a pre-baked architecture to work within it works really well. It's not really what you'd use here, but just saying "this project uses a ports and adapters architecture" can stop it from generating mush by default. I think it's not so much that they're bad at it as that they don't have a clear reason to pick something other than mush. And not just "something" - a specific something, from a fairly short list of architectures suitable for your problem domain.

Re: Claude is not your architect. Stop letting it pretend

#58

Post by joe_mamba »

tquinn35 wrote: post #31
joe_mamba wrote: post #4

Earlier quoted context omitted.

>Code is mediocre in many places. As if code written by devs at major corporations is't mediocre at best. Nokia's Symbian OS took days to build. Days. With a D. Not minutes, not hours but days. One of our devs shipped code to prod with a memory leak thanks to including a library that had "do not use this library in production because it causes a memory leak" written everywhere as warning. So I don't wanna hear about…

I agree with what you’re saying but I think the difference is many managers and above think that AI is infallible or at least much less so than it actually is and that causes problems. Everyone is aware that humans write poor code and treat the code as so. Not so with AI code. I’ve seen devs and managers cut corners in testing/reviewing code cause AI wrote it and they think it’s solid. Sure you could blame anyone cut…

> I think the difference is many managers and above think that AI is infallible

Good for them. I hope they believe this because one of two things will happen.

Either they win on the free market because they went all in on AI and beat their competition thanks to AI productivity increases.

Or, their AI code is shit and they collapse and go bankrupt, and get beaten by the competitors using human written code so then they win on the free market proving AI is useless.

So if AI is good or bad for productivity, the free market will ultimately decide.

My take is that AI is just an amplifier of existing skill. 1x devs using AI can use it to be 10x devs, 10x devs can become 100x devs, while -1x devs will be -10x devs and so on.

Re: Claude is not your architect. Stop letting it pretend

#59

Post by mceachen »

laszlojamf wrote: post #5

I keep hearing that claude is supposedly so agreeable. This doesn't agree with my experience. Claude will often tell me that I'm wrong, and insist on its own solution being right even when I tell it it's wrong.

This is a very recent model behavior change: for me, Opus 4.6, Gemini 3.1 Pro, and ChatGPT 5.4(ish) -- prior models and harnesses suffered much more from sycophancy.

(I still prompt some questions and reviews with "our intern suggested..." to allow models to judge the quality of the content apart from the messenger)

Re: Claude is not your architect. Stop letting it pretend

#60

Post by hluska »

It’s interesting; I haven’t gotten that deep into agentic but use generative AI constantly as a rubber duck that can sometimes come up with something insightful that I missed slash a very enthusiastic junior developer. I generally use chat sessions, often give it specific tasks and then fix anything I don’t quite like. It’s been a great tool, almost like a search engine built for me, but it’s not an architect for me. It’s just a tool and fundamentally, it’s just replaced having dozens of browser tabs open all day.

It’s been quite good for my productivity and the best part for me is that I learn what I’m writing while I’m writing. I can just write things I already understand a lot faster than before. When I work with agentic, I find that I still have to deeply learn the system, but I’ll have to learn it when it falls over instead of at review time.

Post reply on HN