Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is not your architect. Stop letting it pretend

hollandtech.net

Post reply on HN
Print view View original

111–120 of 209 posts

Re: Claude is not your architect. Stop letting it pretend

#111

Post by fn-mote »

AndrewKemendo wrote: post #103
DrewADesign wrote: post #88

Earlier quoted context omitted.

Humans’ general inability to entirely divorce social instincts, responses, and mores while using human language to communicate, especially with something that pantomimes it back, is one of the reasons current chat interfaces are fundamentally flawed. This is working against innate behavior … not something that can be easily switched off. I’ll bet most of the people that can really do it have a hard time intuitively n…

> I’ll bet most of the people that can really do it have a hard time intuitively navigating real social interactions Bingo. Hi that’s me. I’ve been trying to teach people how to use LLMs effectively not just dump shit in them but actually talk to them like you would expect a computer to understand and it totally breaks peoples brains I’m quite successful in helping people get somewhere usable that they weren’t…but to…

> it takes somebody who can hold both extremely large scale problems and very very granular specific implementation problems in your head all at once

This describes the entire software engineering profession to me.

We have come up with all sorts of devices to make this go more smoothly, or to enable us to focus on specific sub-parts as long as possible.

That said, at some point (both in design and integration), you need vision and attention to detail to make progress. The skill seems learnable to me, but watching others struggle sometimes makes me wonder.

Re: Claude is not your architect. Stop letting it pretend

#112

Post by onlyrealcuzzo »

himata4113 wrote: post #110

I have a good story to share that I came across recently. Around 2 years ago I had to clean up a mess because someone who doesn't really know what they're doing designed an instancing system for a game. They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. It took me 2 weeks just to get it to an "a…

> They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue.

Presumably that's better than no game at all.

> It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed.

Doing 2 weeks of fixing might have been hell, but this sounds like it was overall still a great deal for the company.

You're not really selling the "AI is useless story". It might be, but your anecdote seems like just another case of AI being worth it, though obviously flawed.

Re: Claude is not your architect. Stop letting it pretend

#113

Post by himata4113 »

onlyrealcuzzo wrote: post #112
himata4113 wrote: post #110

I have a good story to share that I came across recently. Around 2 years ago I had to clean up a mess because someone who doesn't really know what they're doing designed an instancing system for a game. They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. It took me 2 weeks just to get it to an "a…

> They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. Presumably that's better than no game at all. > It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed. Doing 2 weeks of fixing might have been hell, but this sounds like it was…

No the game failed because of these non stop issues, it lost all hype and the "fixed" version couldn't sustain itself anymore as it took around a month to fix the remaining issues.

At the 2nd company it wasted thousands of dollars of advertisement because the server could not withstand the load and obviously data loss issues tained the image forever and will likely end up the same way.

Also please don't take this as "AI is useless". I use AI and I use it a lot. It's great and I love it. However, without a good understanding of architecture and general development structure you end up with things that can't scale and fail.

Re: Claude is not your architect. Stop letting it pretend

#114

Post by giancarlostoro »

I have probably posted this a zillion times on HN, but tell the model how to work not what you want. If you want it to tell you the how have it write a spec file with links to sources, review the sources, then adjust and approve of the spec file.

Re: Claude is not your architect. Stop letting it pretend

#115

Post by HDThoreaun »

spacedcowboy wrote: post #20
bad_username wrote: post #9

I think the article has the correct message, but I disagree with this: > It’s just incapable of the thing that makes a real architect valuable: saying “no.” From my experience Claude is excellent at saying "no". It won't say "no" if the prompt doesn't call for it (it won't say "no" to your direct request to do something, usually). But it offers good critique and happily pushes back if you make it clear that that's a…

It actually got quite snippy with me, when I was trying to get it to debug some issues. It kept on saying that the "burn rate" wasn't progressing and "we" should refocus our efforts somewhere else. Eventually I got something like "I have told you three times now that this is not the best approach to be taking to reduce the burn-rate and you have not taken that advice". And it stopped helping out. So I was blunt, and…

Dont argue with LLMs. Sometimes they lose the plot, when that happens simply flush the context and start over.

Re: Claude is not your architect. Stop letting it pretend

#116

Post by operatingthetan »

gobdovan wrote: post #85
amarant wrote: post #18

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagin…

> AI is a tool. It needs to be subservient Fun experiment, chat with an LLM and swap roles. Tell it you're gonna be the assistant and them the assisted. I found they're pretty bad at using a human for what they're good for.

I tried it, and the llm gave me an absurd home lab scenario about servers shooting each other in the head to determine which was the "master server". So I told it that it was not an actual problem that it had, and sure enough it admitted it made it up. When you press an llm you will always find there is no internal state behind the thinking. It's just output.

Re: Claude is not your architect. Stop letting it pretend

#118

Post by surgical_fire »

onlyrealcuzzo wrote: post #112
himata4113 wrote: post #110

I have a good story to share that I came across recently. Around 2 years ago I had to clean up a mess because someone who doesn't really know what they're doing designed an instancing system for a game. They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. It took me 2 weeks just to get it to an "a…

> They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. Presumably that's better than no game at all. > It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed. Doing 2 weeks of fixing might have been hell, but this sounds like it was…

> Presumably that's better than no game at all.

Wrong.

A bad game can absolutely tank a studio. Shipping a game that has awful reviews will absolutely affect negatively your sales for future games.

Re: Claude is not your architect. Stop letting it pretend

#119

Post by himata4113 »

surgical_fire wrote: post #118
onlyrealcuzzo wrote: post #112

Earlier quoted context omitted.

> They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. Presumably that's better than no game at all. > It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed. Doing 2 weeks of fixing might have been hell, but this sounds like it was…

> Presumably that's better than no game at all. Wrong. A bad game can absolutely tank a studio. Shipping a game that has awful reviews will absolutely affect negatively your sales for future games.

Luckily this was minecraft so stakes aren't as high, but it definitely tainted the reputation of the company behind it and youtubers will likely never work with them again.

Re: Claude is not your architect. Stop letting it pretend

#120

Post by amarant »

Paracompact wrote: post #61
devin wrote: post #52

Earlier quoted context omitted.

The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction. One reaction to this might be "well that's not what I mean, that suggests you're prompting with too much directionality" which could further be con…

What it actually suggests is that the AI's response to these questions of judgment have little correlation with the thing it's judging. Sure, you can get it to be complimentary, if you want it to be. Sure, you can get it be critical, if you want it to be. But what if I don't know if my design needs to be complimented or critiqued in this instance? This is the default position when seeking input, and so "prompt with m…

So what I do when I'm not sure about something, is I say "I want to achieve X, I was thinking I could solve it by doing Y, what are the pros and cons of this approach, and what is a alternative solution you would suggest?"

And from there it's a interactive discussion drilling down on details until I understand the problem and the solutions better.

It definitely challenges my bias when I do this. The one thing it doesn't challenge is the X. Formulate the problem poorly, and you'll get a bad solution. Or rather, you'll end up with a good solution to the wrong problem. Which is even worse than a bad solution to the right problem.

Which is largely why I'm not at all worried about losing my job to AI. It takes some experience to formulate the problem correctly. I don't feel like I'm made redundant by AI, I'm just way faster than I used to be, my thinking is more abstract.

A good prompt I'll often use is "is there a industry standard solution that is applicable to this problem?" You very rarely want novel solutions. Don't reinvent the wheel just because AI lets you do it 10x as fast. Use a wheel. They're round for a reason.

Sometimes I find it useful to discuss things with a different model. I like Gemini for discussion and Claude for implementation. With Gemini I go about it as a learning session, discussing options and details. I honestly think this is mostly because it compartmentalizes the phases in a natural way for me. One interface for brainstorming and learning, and another for planning and implementing.

Sorry this comment turned into a rather disorganised collection of ramblings, I hope you can extract some kernel of usefulness from it all.

Post reply on HN