Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

451–460 of 544 posts

Re: How I write software with LLMs

#451

Post by visarga »

arialdomartini wrote: post #68
akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

This is anecdotal but just a couple days ago, with some colleagues, we conducted a little experiment to gather that evidence. We used a hierarchy of agents to analyze a requirement, letting agents with different personas (architect, business analyst, security expert, developer, infra etc) discuss a request and distill a solution. They all had access to the source code of the project to work on. Then we provided the v…

An ensemble can spot more bugs / fixes than a single model. I run claude, codex and gemini in parallel for reviews.

Re: How I write software with LLMs

#452

Post by visarga »

Miraste wrote: post #444
arialdomartini wrote: post #68

Earlier quoted context omitted.

This is anecdotal but just a couple days ago, with some colleagues, we conducted a little experiment to gather that evidence. We used a hierarchy of agents to analyze a requirement, letting agents with different personas (architect, business analyst, security expert, developer, infra etc) discuss a request and distill a solution. They all had access to the source code of the project to work on. Then we provided the v…

LLMs also don't have the primary advantage humans get from job separation, diverse perspectives. A council of Opuses are all exploring the exact same weights with the exact same hardware, unlike multiple humans with unique brains and memories. Even with different ones, Codex 5.3 is far more similar to Opus than any two humans are to each other. Telling an Opus agent to focus on security puts it in a different part of…

You can differentiate by context, one sees the work session, the other sees just the code. Same model, but different perspectives. Or by model, there are at least 7 decent models between the top 3 providers.

Re: How I write software with LLMs

#453

Post by visarga »

TheMuenster wrote: post #434
moduspol wrote: post #368

Earlier quoted context omitted.

To me, such techniques feel like temporary cudgels that may or may not even help that will be obsolete in 1-6 months. This is similar to telling Claude Code to write its steps into a separate markdown file, or use separate agents to independently perform many tasks, or some of the other things that were commonly posted about 3-6+ months ago. Now Claude Code does that on its own if necessary, so it's probably a net ne…

Totally agree - the fundamental concept here of automatically improving context control when writing code is absolutely something that will be baked into agents in 6 months. The reason it hasn't yet is mainly because the improvements it makes seem to be very marginal. You can contrast this to something like reasoning, which offered very large, very clear improvements in fundamental performance, and as a result was ta…

Workflow matters too, how you organize your docs, work tasks, reviews. If you do it all by hand you spend a lot of time manually enforcing a process that can be automated.

I think task files with checkable gates are a very interesting animal - they carry intent, plan, work and reviews, at the end of work can become docs. Can be executed, but also passed as value, and reflect on themselves - so they sport homoiconicity and reflexion.

Re: How I write software with LLMs

#454

Post by sarchertech »

raw_anon_1111 wrote: post #410
sarchertech wrote: post #397

Earlier quoted context omitted.

You might have maintained large systems long ago, but if you haven't done it in a while your skill atrophies. And the most important part is you haven't maintained any large systems written by AI, so stating that they will work is nonsense. I won't state that AI can't get better. AI agents might replace all of us in the future. But what I will tell you is based on my experience and reasoning I have very strong doubts…

Well seeing that Claude code was just introduced last year - it couldn’t have been that long since I didn’t code with AI. And did I mention I got my start working in cloud consulting as a full time blue badge, RSU earning employee at a little company you might have heard of based in Seattle? So since I have worked at the second largest employee in the US, unless you have worked for Walmart - I don’t think you have wo…

>Well seeing that Claude code was just introduced last year - it couldn’t have been that long since I didn’t code with AI.

That's my entire point!

>And unless your experience coding is before 1986 when I was coding in assembly language in 65C02 as a hobby, I think I might have a wee bit more than you.

Yeah a real wee bit. I started in the late 80s in Tandy Basic.

>I think you should probably save your “I have more experience” for someone who hasn’t been doing this professionally for 30 years for everything from startups, to large enterprises, to BigTech.

I never said anything about having more experience than you, but I've been doing this almost as long as you have. Also at everywhere from startups to large enterprises to BigTech.

But relevant to the discussion at hand, I haven't been consulting for the last part of my career where I could just lob something over the fence and walk away before I have to deal with the consequences of my decisions. This is what seems to be coloring your experience.

Re: How I write software with LLMs

#455

Post by sarchertech »

LogicFailsMe wrote: post #408
sarchertech wrote: post #399

Earlier quoted context omitted.

I don't think society is going to collapse because of AI because I don't think the current architectures have any chance of becoming AGI. I think that if AGI is even something we're capable of it's very far off. I think that if CEOs can replace us soon, it's because AGI got here much sooner than I predicted. And if that happens we have 2 options Mad Max and Star Trek and Mad Max is the more likely of the 2.

What's with all the catastrophic thinking then? Mad Max? Collapse of Society because 45% unemployment? I really hate people on principle but I have more faith in them looking out for their own self interest than you do apparently. Mad Max specifically requires a ridiculous amount of intact infrastructure for all the gasoline (you know gasoline goes bad in 3-6 months? Yeah didn't think so), manufacturing for all the p…

>Mad Max specifically requires a ridiculous amount of intact infrastructure for all the gasoline (you know gasoline goes bad in 3-6 months? Yeah didn't think so)

Is this a joke or do you have a learning disability?

>But I expect 24/7 IC3 level engineering as a service for $200/month to be more than enough and I think that's a year or two away. And you can either prepare for that or scream how the sky is falling, your choice.

Or I could do neither and write you off as a gasbag who doesn't know what he's talking about like all the other ex-amazon management I've had the pleasure to work with over the years.

Re: How I write software with LLMs

#456

Post by sarchertech »

raw_anon_1111 wrote: post #407
sarchertech wrote: post #385

Earlier quoted context omitted.

The difference is that a human is that a human can reason about their code changes to a much higher degree than an AI can. If you don't think this is true and you think we're working with AGI, why would you bother architecting anything all or building in any guard rails. Why not just feed the AI the text of the contract your working from and let it rip.

You give way too much credit to the average mid level ticket taker. And again, why do I care how the code does it as long as it meets the functional and none functional requirements?

Because in a real application with real users all of the functional and non-functional requirements aren't documented anywhere but in the code.

Re: How I write software with LLMs

#457

Post by sarchertech »

raw_anon_1111 wrote: post #409
sarchertech wrote: post #377

Earlier quoted context omitted.

Not that humans can't write 100k line programs bug free or without running parts of it. An AI cannot write a 100k line program on its own without external guard rails otherwise it spins out. This has nothing to do with whether the agent is allowed to run the code itself. This is well documented. Look at what was required to allow Claude to write a "C compiler". This has nothing to do with whether it's bug free. It li…

Absolutely no one is arguing that you shouldn’t have a combination of manual and automated tests around either AI or human generated code or that you shouldn’t have a thoughtful design

In a non-trivial app you can't test your way through all of the e2e workflows and thoughtful design isn't what I'm talking about.

How many bugs have you seen that passed your automated and manual testing? Probably 99.9% of them.

Now imagine that you take those same test suites and you unleash an agent on the code that has far worse reasoning capabilities than a human and you tell them they can change anything in the code as long as the tests pass.

Re: How I write software with LLMs

#458

Post by sarchertech »

anthonyrstevens wrote: post #361
jplusequalt wrote: post #329

Earlier quoted context omitted.

Classic HN-ism. To focus on the semantics of a statement while ignoring the greater point in order to argue why someone is wrong.

I think it's a perfectly fine point. The OP said (my interpretation) that LLMs are messy, non-deterministic, and can produce bad code. The same is true of many humans, even those whose "job" is to produce clean, predictable, good code. The OP would like the argument to be narrowly about LLMs, but the bigger point even is "who generates the final code, and why and how much do we trust them?"

As of right now agents have almost no ability to reason about the impact of code changes on existing functionality.

A human can produce a 100k LOC program with absolute no external guardrails at all. An agent Can't do that. To produce a 100k LOC program they require external feedback forcing them from spiraling off into building something completely different.

This may change. Agents may get better.

Re: How I write software with LLMs

#459

Post by raw_anon_1111 »

sarchertech wrote: post #457
raw_anon_1111 wrote: post #409

Earlier quoted context omitted.

Absolutely no one is arguing that you shouldn’t have a combination of manual and automated tests around either AI or human generated code or that you shouldn’t have a thoughtful design

In a non-trivial app you can't test your way through all of the e2e workflows and thoughtful design isn't what I'm talking about. How many bugs have you seen that passed your automated and manual testing? Probably 99.9% of them. Now imagine that you take those same test suites and you unleash an agent on the code that has far worse reasoning capabilities than a human and you tell them they can change anything in the…

So if bugs pass through testing which they have forever, wouldn’t that imply that humans are just as fallible as AI - and slower?

I never suggested letting agents code for a day on end. I use AI to code well defined tasks and treat it like a mid level ticket taker

Re: How I write software with LLMs

#460

Post by mbesto »

danbruc wrote: post #297
mbesto wrote: post #285

Earlier quoted context omitted.

> and that is not great code When you say "is not great code" can you elaborate? Does the code work or not?

I don't know, I would assume it works but I would not expect it to be free of bugs. But that is the baseline for code, being correct - up to some bugs - is the absolute minimum requirement, code quality starts from there - is it efficient, is it secure, is it understandable, is it maintainable, ...

So do you expect it not to be free of bugs because you've run a comprehensive test on it, read all of the code yourself or are you just concluding that because you know it was generated by an LLM?
Post reply on HN