Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

131–140 of 544 posts

Re: How I write software with LLMs

#131

Post by sigbottle »

lbreakjai wrote: post #109
akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

The different models is a big one. In my workflow, I've got opus doing the deep thinking, and kimi doing the implementation. It helps manage costs. Sample size of one, but I found it helps guard against the model drifting off. My different agents have different permissions. The worker can not edit the plan. The QA or planner can't modify the code. This is something I sometimes catch codex doing, modifying unrelated s…

I recently had a horrible misalignment issue with a 1 agent loop. I've never done RL research, but this kind of shit was the exact kind of thing I heard about in RL papers - shimming out what should be network tests by echoing "completed" with the 'verification' being grepping for "completed", and then actually going and marking that off as "done" in the plan doc...

Admittedly I was using gsdv2; I've never had this issue with codex and claude. Sure, some RL hacking such as silent defaults or overly defensive code for no reason. Nothing that seemed basically actively malicious such as the above though. Still, gsdv2 is a 1-agent scaffolding pipeline.

I think the issue is that these 1-agent pipelines are "YOU MUST PLAN IMPLEMENT VERIFY EVERYTHING YOURSELF!" and extremely aggressive language like that. I think that kind of language coerces the agent to do actively malicious hacks, especially if the pipeline itself doesn't see "I am blocked, shifting tasks" as a valid outcome.

1-agent pipelines are like a horrible horrible DFS. I still somewhat function when I'm in DFS mode, but that's because I have longer memory than a goldfish.

Re: How I write software with LLMs

#132

Post by lbreakjai »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

I just don't want to build the habit of being a sloppy writer, because it will eventually leak into the conversations I have with real humans.

Re: How I write software with LLMs

#133

Post by est »

codeflo wrote: post #59
est wrote: post #47

Earlier quoted context omitted.

> the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? There's a 63 pages paper with mathematical proof if you really into this. https://arxiv.org/html/2601.03220v1 My takeaway: AI learns from real-world texts, and real-world corpus are used to have a role split of architect/developer/reviewer

>> the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? > There's a 63 page paper with mathematical proof if you really into this. > https://arxiv.org/html/2601.03220v1 I'm confused. The linked paper is not primarily a mathematics paper, and to the extent that it is, proves nothing remotely like the question that was asked.

> proves nothing remotely like the question that was asked

I am not an expert, but by my understanding, the paper prooves that a computationally bounded "observer" may fail to extract all the structure present in the model in one computation. aka you can't always one-shot perfect code.

However, arrange many pipelines of roles "observers" may gradually get you there

Re: How I write software with LLMs

#134

Post by takwatanabe »

We build and run a multi-agent system. Today Cursor won. For a log analysis task — Cursor: 5 minutes. Our pipeline: 30 minutes.

Still a case for it: 1. Isolated contexts per role (CS vs. engineering) — agents don't bleed into each other 2. Hard permission boundaries per agent 3. Local models (Qwen) for cheap routine tasks

Multi-agent loses at debugging. But the structure has value.

Re: How I write software with LLMs

#135

Post by wiseowise »

silisili wrote: post #15
borski wrote: post #12

Earlier quoted context omitted.

LLMs can build anything. The real question is what is worth building, and how it’s delivered. That is what is still human. LLMs, by nature of not being human, cannot understand humans as well as other humans can. (See every attempt at using an LLM as a therapist) In short: LLMs will eventually be able to architect software. But it’s still just a tool

What is the use of software eng/architect at that point? It's a tool, but one that product or C levels can use directly as I see it?

> It's a tool, but one that product or C levels can use directly as I see it?

Wait, I thought product and C level people are so busy all the time that they can’t fart without a calendar invite, but now you say they have time to completely replace whole org of engineers?

Re: How I write software with LLMs

#136

Post by vikramkr »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

For models that reveal reasoning traces I've seen their inner nature as a word calculator show up as they spend way too many tokens complaining about the typo (and AI code review bots also seem obsessed with typos to the point where in a mid harness a few too many irrelevant typos means the model fixates on them and doesn't catch other errors). I don't know if they've gotten better at that recently but why bother. Plus there's probably something to the model trying to match the user's style (it is auto complete with many extra steps) resulting in sloppier output if you give it a sloppier prompt.

Re: How I write software with LLMs

#137

Post by wiseowise »

borski wrote: post #58
mattmanser wrote: post #56

Earlier quoted context omitted.

Where's this delusion come from recently that great engineers didnt write code? What a load of crap. All you're doing is describing a different job role. What you're talking about is BA work, and a subset of engineers are great at it, but most are just ok. You're claiming a part of the job that was secondary, and not required, is now the whole job.

I never said great engineers didn’t write code. But writing the code was never the point. The point has always been delivering the product to the customer, in any industry. Code is rarely the deliverable. That’s my point.

> But writing the code was never the point.

Is that why most prestigious jobs grilled you like a devil on algos/system design?

> The point has always been delivering the product to the customer, in any industry. Code is rarely the deliverable.

That’s just nonsense. It’s like saying “delivering product was always the most important thing, not drinking water”.

Re: How I write software with LLMs

#138

Post by never_inline »

akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

I think this is just anthropomorphism. Sub agents make sense as a context saving mechanism.

Aider did an "architect-editor" split where architect is just a "programmer" who doesn't bother about formatting the changes as diff, then a weak model converts them into diffs and they got better results with it. This is nothing like human teams though.

Re: How I write software with LLMs

#139

Post by danbruc »

I randomly clicked and scrolled through the source code of Stavrobot - The largest thing I’ve built lately is an alternative to OpenClaw that focuses on security. [1] and that is not great code. I have not used any AI to write code yet but considered trying it out - is this the kind of code I should expect? Or maybe the other way around, has someone an example of some non-trivial code - in size and complexity - written by an AI - without babysitting - and the code being really good?

[1] https://github.com/skorokithakis/stavrobot

Re: How I write software with LLMs

#140

Post by wiseowise »

codeflo wrote: post #71

I know the argument I'm going to make is not original, but with every passing week, it's becoming more obvious that if the productivity claims were even half true, those "1000x" LLM shamans would have toppled the economy by now. Were are the slop-coded billion dollar IPOs? We should have one every other week.

They’re busy writing applications for their dogs and building “jerk me off” functionality into their OpenClaw fork. Once they’re done you’ll be sorry you ever asked.
Post reply on HN