Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

151–160 of 544 posts

Re: How I write software with LLMs

#151

Post by miguelgrinberg »

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results.

It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't.

In my experience the differences are mostly in how the code produced by the LLM is reviewed. Developers who have experience reviewing code are more likely to find problems immediately and complain they aren't getting great results without a lot of hand holding. And those who rarely or never reviewed code from other developers are invariably going to miss stuff and rate the output they get higher.

Re: How I write software with LLMs

#152

Post by jwilliams »

akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

If you know what you need, my experience is that a well-formed single-prompt that fits the context gives the best results (and fastest).

If you’re exploring an idea or iterating, the roles can help break it down and understand your own requirements. Personally I do that “away” from the code though.

Re: How I write software with LLMs

#153

Post by xnorswap »

jstanley wrote: post #130
xnorswap wrote: post #117

Earlier quoted context omitted.

I agree, it's just easier to write requirements and refine things as if writing with a human. I no longer care that it risks anthropomorphising it, as that fight has long been lost. I prefer to focus on remembering it doesn't actually think/reason than not being polite to it. Keeping everything generally "human readable" also the advantage of it being easier for me to review later if needed.

How can you use these models for any length of time and walk away with the understanding that they do not think or reason? What even is thinking and reasoning if these models aren't doing it?

They produce wonderful results, they are incredibly powerful, but they do not think or reason.

Among many other factors, perhaps the most key differentiator for me that prevents me describing these as thinking, is proactivity.

LLMs are never pro-active.

( No, prompting them on a loop is not pro-activity ).

Human brains are so proactive that given zero stimuli they will hallucinate.

As for reasoning, they simply do not. They do a wonderful facsimile of reasoning, one that's especially useful for producing computer code. But they do not reason, and it is a mistake to treat them as if they can.

Re: How I write software with LLMs

#154

Post by claud_ia »

kleiba wrote: post #57

I write very little code these days, so I've been following the AI development mostly from the backseat. One aspect I fail to grasp perfectly is what the practical differences are between CLI (so terminal-based) agents and ones fully integrated into an IDE. Could someone chime in and give their opinion on what are the pros and cons of either approach?

[dead]

Re: How I write software with LLMs

#155

Post by danbruc »

never_inline wrote: post #145
danbruc wrote: post #139

I randomly clicked and scrolled through the source code of Stavrobot - The largest thing I’ve built lately is an alternative to OpenClaw that focuses on security. [1] and that is not great code. I have not used any AI to write code yet but considered trying it out - is this the kind of code I should expect? Or maybe the other way around, has someone an example of some non-trivial code - in size and complexity - writt…

I would suggest not delegating the LLD (class / interface level design) to the LLM. The clankeren are super bad at it. They treat everything as a disposable script. Also document some best practices in AGENT.md or whatever it's called in your app. Eg * All imports must be added on top of the file, NEVER inside the function. * Do not swallow exceptions unless the scenario calls for fault tolerance. * All functions nee…

What actually stood out to me is how bad the functions are, they have no structure. Everything just bunched together, one line after the other, whatever it is, and almost no function calls to provide any structure. And also a ton of logging and error handling mixed in everywhere completely obscuring the actual functionality.

EDIT: My bad, the code eventually calls into dedicated functions from database.ts, so those 200 lines are mostly just validation and error handling. I really just skimmed the code and the amount of it made me assume that it actually implements the functionality somewhere in there.

Example, Agent.ts, line 93, function createManageKnowledgeTool() [1]. I would have expected something like the following and not almost 200 lines of code implementing everything in place. This also uses two stores of some sort - memory and scratchpad - and they are also not abstracted out, upsert and delete deal with both kinds directly.

  switch (action)
  {
    case "help":
      return handleHelpAction(arguments);

    case "upsert":
      return handleUpsertAction(arguments);

    case "delete":
      return handleDeleteAction(arguments);

    default:
      return handleUnknowAction(arguments);
  }
[1] https://github.com/skorokithakis/stavrobot/blob/master/src/a...

Re: How I write software with LLMs

#156

Post by zingar »

kleiba wrote: post #57

I write very little code these days, so I've been following the AI development mostly from the backseat. One aspect I fail to grasp perfectly is what the practical differences are between CLI (so terminal-based) agents and ones fully integrated into an IDE. Could someone chime in and give their opinion on what are the pros and cons of either approach?

I guess you’re probably looking for someone who uses cursor etc to answer but here’s a data point from someone a bit off the beaten path.

My editor supports both modes (emacs). I have the editor integration features (diff support etc) turned off and just use emacs to manage 5+ shells that each have a CLI agent (one of Claude, opencode, amp free) running in them.

If I want to go deep into a prompt then I’ll write a markdown file and iterate on it with a CLI.

Re: How I write software with LLMs

#157

Post by arialdomartini »

titanomachy wrote: post #144
arialdomartini wrote: post #68

Earlier quoted context omitted.

This is anecdotal but just a couple days ago, with some colleagues, we conducted a little experiment to gather that evidence. We used a hierarchy of agents to analyze a requirement, letting agents with different personas (architect, business analyst, security expert, developer, infra etc) discuss a request and distill a solution. They all had access to the source code of the project to work on. Then we provided the v…

If it could be done with 30 cents of Haiku calls, maybe it wasn't a complicated enough project to provide good signal?

Fair point. I could try with a harder problem. This still does not explain why Claude Code felt the need to use Opus, and why Opus felt the need to burn 12$ or such an easy task. I mean, it's 40 times the cost.

Re: How I write software with LLMs

#158

Post by peterweisz »

Great article. I'd recommmend to make guardrails and benchmarking an integral part of prompt engineering. Think of it as kind of a system prompt to your Opus 4.6 architect: LangChain, RAG, LLm-as-a-judge, MCP. When I think about benchmarks I always ask it to research for external DB or other ressources as a referencing guardrail

Re: How I write software with LLMs

#159

Post by ChrisGreenHeur »

ashwinsundar wrote: post #11
plastic041 wrote: post #9

I wanted to know how to make softwares with LLM "without losing the benefit of knowing how the entire system works" and "intimately familiar with each project’s architecture and inner workings", while "have never even read most of their code". (Because obviously, you can't.) But OP didn't explain that. You tell LLM to create something, and then use another LLM to review it. It might make the result safer, but it does…

Hot take: you can't have your cake and eat it too. If you aren't writing code, designing the system, creating architecture, or even writing the prompt, then you're not understanding shit. You're playing slots with stochastic parrots The code grows beyond my usual comprehension, I'd have to really read through it for a while. Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until i…

the hardware you typed this on was designed by hardware architects that write little to no code. just types up a spec to be implemented by verilog coders.

Re: How I write software with LLMs

#160

Post by or_am_i »

miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

It's always easier to blame the model and convince yourself that you have some sort of talent in reviewing LLM's work that others don't.

In my experience the differences are mostly in how the code produced by LLM is prompted and what context is given to the agent. Developers who have experience delegating their work are more likely to prevent downstream problems from happening immediately and complain their colleagues cannot prompt as efficiently without a lot of hand holding. And those who rarely or never delegated their work are invariably going to miss crucial context details and rate the output they get lower.

Post reply on HN