Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

211–220 of 544 posts

Re: How I write software with LLMs

#211

Post by christofosho »

ttanveer wrote: post #163
miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

That seems to make sense. Any suggestions to improve this skill of reviewing code? I think especially a number of us more junior programmers lack in this regard, and don't see a clear way of improving this skill beyond just using LLMs more and learning with time?

Find another developer and pair/work together on a project. It doesn't need to be serious, but you should organize it like it is. So, a breakdown of tasks needed to accomplish the goal first. And then many pull requests into the source that can be peer reviewed.

Re: How I write software with LLMs

#212

Post by chriswarbo »

cpt_sobel wrote: post #84
qsera wrote: post #81

Earlier quoted context omitted.

I think it is easier to be polite always and not switch between polite and non-polite mode depending on who you are talking to.

I get what you're saying, but I'm not talking about swearing at the model or anything, I'm only implying that investing energy in formulating a syntactically nice sentence doesn't or shouldn't bring any value, and that I don't care if I hurt the model's feelings (it doesn't have any). Note, why would the author write "Email will arrive from a webhook, yes." instead of "yy webhook"? In the second case I wouldn't be im…

> investing energy in formulating a syntactically nice sentence

This seem to be completely subjective; I write syntactically/grammatically "nice" sentences to LLMs, because that's how I write. I would have to "invest energy" to force myself to write in that supposedly "simpler" style.

Re: How I write software with LLMs

#213

Post by __alexs »

miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

I review most of the code I get LLMs to write and actually I think the main challenge is finding the right chunk size for each task you ask it to do.

As I use it more I gain more intuition about the kinds of problems it can handle on it's, vs those that I need to work on breaking down into smaller pieces before setting it loose.

Without research and planning agents are mostly very expensive and slow to get things done, if they even can. However with the right initial breakdown and specification of the work they are incredibly fast.

Re: How I write software with LLMs

#214

Post by ilovetux »

I like the approach outlined in the article. These days having a roadmap for yourself while cruising at highway speeds helps make sense of the chaos.

One big pain point that has existed forever and has never really been addresses adequately is the ability to come up with requirements.

Sure, it sounds easy, I need the app to do x, y and z. But requirements change in real time because of lack of foresight, change of business needs, an unexpected roadblock and more contribute to changing requirements.

So, the advice to come up with the requirements by yourself or with the LLM miss the biggest pain point.

I'd like to see a resurgence of flow charts, IPO (Input, Processing and Output) charts and other tools to organize requirements spring up to help with really nailing down requirements.

I will say, though, some of the pain is relieved because the agent can perform a huge refactor in a couple of minutes, but that opens a whole new can of worms.

Re: How I write software with LLMs

#215

Post by chriswarbo »

stavros wrote: post #205
christofosho wrote: post #2

I like reading these types of breakdowns. Really gives you ideas and insight into how others are approaching development with agents. I'm surprised the author hasn't broken down the developer agent persona into smaller subagents. There is a lot of context used when your agent needs to write in a larger breadth of code areas (i.e. database queries, tests, business logic, infrastructure, the general code skeleton). I'v…

I don't think that splitting into subagents that use the same model will really help. I need to clarify this in the post, but the split is 1) so I can use Sonnet to code and save on some tokens and 2) so I can get other models to review, to get a different perspective. It seems to me that splitting into subagents that use the same model is kind of like asking a person to wear three different hats and do three differe…

I'm considering using subagents, as a way to manage context and delegate "simple" tasks to cheaper models (if you want to see tokens burn, watch Opus try fixing a misplaced ')' in a Lisp file!).

I see what you mean w.r.t. different hats; but is it useful to have different tools available? For example, a "planner" having Web access and read-only file access, versus a "developer" having write access to files but no Web access?

Re: How I write software with LLMs

#216

Post by input_sh »

baxtr wrote: post #183
miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

I thought I try to debunk your argument with a food example. I am not sure I succeeded though. Judge for yourself: It's always easier to blame the ingredients and convince yourself that you have some sort of talent in how you cook that others don't. In my experience the differences are mostly in how the dishes produced in the kitchen are tasted. Chefs who have experience tasting dishes critically are more likely to f…

[dead]

Re: How I write software with LLMs

#217

Post by TacticalCoder »

danbruc wrote: post #139

I randomly clicked and scrolled through the source code of Stavrobot - The largest thing I’ve built lately is an alternative to OpenClaw that focuses on security. [1] and that is not great code. I have not used any AI to write code yet but considered trying it out - is this the kind of code I should expect? Or maybe the other way around, has someone an example of some non-trivial code - in size and complexity - writt…

> is this the kind of code I should expect?

Sadly yes. But it "works", for some definition of working. We all know it's going to be a maintenance nightmare seen the gigantic amount of code and projects now being generated ad infinitum. As someone commented in this thread: it can one-shot an app showing restaurant locations on a map and put a green icon if they're open. But don't except good code, secure code, performant code and certainly not "maintainable code".

By definition, unless the AIs can maintain that code, nothing is maintainable anymore: the reason being the sheer volume. Humans who could properly review and maintain code (and that's not many) are already outnumbered.

And as more and more become "prompt engineers" and are convinced that there's no need to learn anything anymore besides becoming a prompt engineer, the amount of generated code is only going to grow exponentially.

So to me it is the kind of code you should expect. It's not perfect. But it more or less works. And thankfully it shouldn't get worse with future models.

What we now need is tools, tools and more tools: to help keep these things on tracks. If we ever to get some peace of mind about the correctness of this unreviewable generate code, we'll need to automate things like theorem provers and code coverage (which are still nowhere to be seen).

And just like all these models are running on Linux and QEMU and Docker (dev container) and heavily using projects like ripgrep (Claude Code insist on having ripgrep installed), I'm pretty sure all these tools these models rely on and shall rely on to produce acceptable results are going to be, very mostly, written by humans.

I don't know how to put it nicely: an app showing green icon next to open restaurants on a map ain't exactly software to help lift off a rocket or to pilot a MRI machine.

BTW: yup, I do have and use Claude Code. Color me both impressed and horrified by the "working" amount un unmaintainable mess it can spout. Everybody who understands something about software maintenance should be horrified.

Re: How I write software with LLMs

#218

Post by hellosimon »

or_am_i wrote: post #160
miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

It's always easier to blame the model and convince yourself that you have some sort of talent in reviewing LLM's work that others don't. In my experience the differences are mostly in how the code produced by LLM is prompted and what context is given to the agent. Developers who have experience delegating their work are more likely to prevent downstream problems from happening immediately and complain their colleague…

Partly true, but I think there's a real skill in catching subtle logic errors in generated code too not just prompting well. Both matter.

Re: How I write software with LLMs

#219

Post by dgb23 »

jstanley wrote: post #130
xnorswap wrote: post #117

Earlier quoted context omitted.

I agree, it's just easier to write requirements and refine things as if writing with a human. I no longer care that it risks anthropomorphising it, as that fight has long been lost. I prefer to focus on remembering it doesn't actually think/reason than not being polite to it. Keeping everything generally "human readable" also the advantage of it being easier for me to review later if needed.

How can you use these models for any length of time and walk away with the understanding that they do not think or reason? What even is thinking and reasoning if these models aren't doing it?

Thinking and reasoning cannot be abstracted away from the individual who experiences the thinking and reasoning itself and changes because of it.

LLMs are amazing, but they represent a very narrow slice of what thinking is. Living beings are extremely dynamic and both much more complex and simple at the same time.

There is a reason for:

- companies releasing new versions every couple of months

- LLMs needing massive amounts of data to train on that is produced by us and not by itself interacting with the world

- a massive amount of manual labor being required both for data labeling and for reinforcement learning

- them not being able to guide through a solution, but ultimately needing guidance at every decision point

Re: How I write software with LLMs

#220

Post by therealpygon »

miguelgrinberg wrote: post #151

> One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Develope…

I think that entirely disregarding the fundamental operation of LLMs with dismissiveness is ungrounded. You are literally saying it isn’t a skill issue while pointing out a different skill issue.

It is absolutely, unequivocally, patently false to say that the input doesn’t affect the output, and if the input has impact, then it IS a skill.

Post reply on HN