Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

91–100 of 544 posts

Re: How I write software with LLMs

#91

Post by nvardakas »

arialdomartini wrote: post #68
akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

This is anecdotal but just a couple days ago, with some colleagues, we conducted a little experiment to gather that evidence. We used a hierarchy of agents to analyze a requirement, letting agents with different personas (architect, business analyst, security expert, developer, infra etc) discuss a request and distill a solution. They all had access to the source code of the project to work on. Then we provided the v…

[dead]

Re: How I write software with LLMs

#92

Post by Havoc »

akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

Yeah always seemed pretty sus to me to.

At the same time I can see a more linear approach doing similar. Like when I ask for an implementation plan that is functional not all that different from an architect agent even if not wrapped in such a persona

Re: How I write software with LLMs

#93

Post by jstanley »

cpt_sobel wrote: post #84
qsera wrote: post #81

Earlier quoted context omitted.

I think it is easier to be polite always and not switch between polite and non-polite mode depending on who you are talking to.

I get what you're saying, but I'm not talking about swearing at the model or anything, I'm only implying that investing energy in formulating a syntactically nice sentence doesn't or shouldn't bring any value, and that I don't care if I hurt the model's feelings (it doesn't have any). Note, why would the author write "Email will arrive from a webhook, yes." instead of "yy webhook"? In the second case I wouldn't be im…

"yy webhook" is much less clear. It could just as easily mean "why webhook" as "yes webhook".

It's also actually more trouble to formulate abbreviated sentences than normal ones, at least for literate adults who can type reasonably well.

Re: How I write software with LLMs

#94

Post by giuscri »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

one reason to do that could be it’s trained on conversations happened between humans.

Re: How I write software with LLMs

#95

Post by jstummbillig »

cpt_sobel wrote: post #80
raincole wrote: post #77

Earlier quoted context omitted.

Because some people like to be polite? Is it this hard to understand? Your hand-written prompts are unlikely to take significant chunk of context window anyway.

Polite to whom?

Anything or anyone. Being polite to your surroundings reflects in your surroundings.

Re: How I write software with LLMs

#96

Post by cpt_sobel »

mrbungie wrote: post #89
tarsinge wrote: post #83

Earlier quoted context omitted.

The reasoning is by being polite the LLM is more likely to stay on a professional path: at its core a LLM try to make your prompt coherent with its training set, and a polite prompt + its answer will score higher (gives better result) than a prompt that is out of place with the answer. I understand to some people it could feel like anthropomorphising and could turn them off but to me it's purely about engineering. Ed…

I remember studies that showed that being mean with the LLM got better answers, but by the other hand I also remember an study showing that maximizing bug-related parameters ended up with meaner/malignant LLMs.

Surely this could depend on the model, and I'm only hypothesizing here, but being mean (or just having a dry tone) might equal a "cut the glazing" implicit instruction to the model, which would help I guess.

Re: How I write software with LLMs

#97

Post by Havoc »

jumploops wrote: post #20

This is similar to how I use LLMs (architect/plan -> implement -> debug/review), but after getting bit a few times, I have a few extra things in my process: The main difference between my workflow and the authors, is that I have the LLM "write" the design/plan/open questions/debug/etc. into markdown files, for almost every step. This is mostly helpful because it "anchors" decisions into timestamped files, rather than…

Yeah same. The markdown thing also helps with the multi model thing. Can wipe context and have another model look at the code and markdown plan with fresh eyes easily

Re: How I write software with LLMs

#98

Post by dgb23 »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

I can't speak for everyone, but to me the most accurate answer is that I'm role-playing, because it just flows better.

In the back of my head I know the chatbot is trained on conversations and I want it to reflect a professional and clear tone.

But I usually keep it more simple in most cases. Your example:

> I'd like to add email support to this bot. Let's think through how we would do this.

I would likely write as:

> if i wanted to add email support, how would you go about it

or

> concise steps/plan to add email support, kiss

But when I'm in a brainstorm/search/rubber-duck mode, then I write more as if it was a real conversation.

Re: How I write software with LLMs

#99

Post by kqr »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

I think it mattered a lot more a few years ago, when the user's prompts were almost all context the LLM had to go by. A prompt written in a sloppy style would cause the LLM to respond in a sloppy style (since it's a snazzy autocomplete at its core). LLMs reason in tokens, so a sloppy style leads it to mimic the reasoning that it finds in the sloppy writing of its training data, which is worse reasoning.

These days, the user prompt is just a tiny part of the context it has, so it probably matters less or not at all.

I still do it though, much like I try to include relevant technical terminology to try to nudge its search into the right areas of vector space. (Which is the part of the vector space built from more advanced discourse in the training material.)

Re: How I write software with LLMs

#100

Post by well_ackshually »

cpt_sobel wrote: post #84
qsera wrote: post #81

Earlier quoted context omitted.

I think it is easier to be polite always and not switch between polite and non-polite mode depending on who you are talking to.

I get what you're saying, but I'm not talking about swearing at the model or anything, I'm only implying that investing energy in formulating a syntactically nice sentence doesn't or shouldn't bring any value, and that I don't care if I hurt the model's feelings (it doesn't have any). Note, why would the author write "Email will arrive from a webhook, yes." instead of "yy webhook"? In the second case I wouldn't be im…

>investing energy

For the vast majority of people, using capital letters and saying please doesn't consume energy, it just is. There's a thousand things in your day that consume more energy like a shitty 9AM daily.

Post reply on HN