Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I write software with LLMs

stavros.io

Post reply on HN
Print view View original

101–110 of 544 posts

Re: How I write software with LLMs

#101

Post by koe123 »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

Just stream of consciousness into the context window works wonders for me. More important to provide the model good context for your question

Re: How I write software with LLMs

#102

Post by TheDong »

cpt_sobel wrote: post #88
tarsinge wrote: post #83

Earlier quoted context omitted.

The reasoning is by being polite the LLM is more likely to stay on a professional path: at its core a LLM try to make your prompt coherent with its training set, and a polite prompt + its answer will score higher (gives better result) than a prompt that is out of place with the answer. I understand to some people it could feel like anthropomorphising and could turn them off but to me it's purely about engineering. Ed…

> If the result of your prompt + its answer it's more likely to score higher i.e. gives better result that a prompt that feels out of place with the answer Sure seems like this could be the case with the structure of the prompt, but what about capitalizing the first letter of sentence, or adding commas, tag questions etc? They seem like semantics that will not play any role at the end

Why wouldn't capitalization, commas, etc do well?

These are text completion engines.

Punctuation and capitalization is found in polite discussion and textbooks, and so you'd expect those tokens to ever so slightly push the model in that direction.

Lack of capitalization pushes towards text messages and irc perhaps.

We cannot reason about these things in the same way we can reason about using search engines, these things are truly ridiculous black boxes.

Re: How I write software with LLMs

#103

Post by trq01758 »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

My view is that when some "for bots only" type of writing becomes a habit, communication with humans will atrophy. Tokens be damned, but this kind of context switch comes at much too high a cost.

Re: How I write software with LLMs

#104

Post by dgb23 »

nacozarina wrote: post #90
cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

agree, prompting a token predictor like you’re talking to a person is counterproductive and I too wish it would stop the models consistently spew slop when one does it, I have no idea where positive reinforcement for that behavior is coming from

[dead]

Re: How I write software with LLMs

#105

Post by kqr »

thenthenthen wrote: post #29

Haha love the Sleight of hand irregular wall clock idea. I once had a wall clock where the hand showing the seconds would sometimes jump backwards, it was extremely unsettling somehow because it was random. It really did make me question my sanity.

This used to be one of my recurring nightmares when I was a child. The three I remember were (1) clocks suddenly starting to go backwards, either partially or completely; (2) radio turning on without being able to turn it off, and (3) house fire. There really is something about clocks.

Re: How I write software with LLMs

#106

Post by lbreakjai »

It's interesting to see some patterns starting to emerge. Over time, I ended up with a similar workflow. Instead of using plan files within the repository, I'm using notion as the memory and source of truth.

My "thinker" agent will ask questions, explore, and refine. It will write a feature page in notion, and split the implementation into tasks in a kanban board, for an "executor" to pick up, implement, and pass to a QA agent, which will either flag it or move it to human review.

I really love it. All of our other documentation lives in notion, so I can easily reference and link business requirements. I also find it much easier to make sense of the steps by checking the tickets on the board rather than in a file.

Reviewing is simpler too. I can pick the ticket in the human review column, read the requirements again, check the QA comments, and then look at the code. Had a lot of fun playing with it yesterday, and I shared it here:

https://github.com/marcosloic/notion-agent-hive

Re: How I write software with LLMs

#107

Post by vbezhenar »

slopinthebag wrote: post #62
jaredklewis wrote: post #37

Earlier quoted context omitted.

> what's the evidence What’s the evidence for anything software engineers use? Tests, type checkers, syntax highlighting, IDEs, code review, pair programming, and so on. In my experience, evidence for the efficacy of software engineering practices falls into two categories: - the intuitions of developers, based in their experiences. - scientific studies, which are unconvincing. Some are unconvincing because they atte…

Most developer intuitions are wrong. See: OOP

Intuition is subjective. It's hard to convert subjective experience to objective facts.

Re: How I write software with LLMs

#108

Post by dmos62 »

cpt_sobel wrote: post #72

In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this part…

I choose to talk in a respectful way, because that's how I want to communicate: it's not because I'm afraid of retaliation or burning bridges. It's because I am caring and conscious. If I think that something doesn't have feelings or long-term memory, whether it's AI or a piece of rock on the side of a trail, it in no way leads me to be abusive to it.

Further, an LLM being inherently sycophantic leads to it mimmicking me, so if I talk to it in a stupid or abusive (which is just another form of stupidity, in my eyes) manner, it will behave stupid. Or, that's what I'd expect. I've not researched this in a focused way, but I've seen examples where people get LLMs to be very unintelligent by prompting riddles or intelligence tests in highly-stylized speech. I wanted to say "highly-stupid speech", but "stylized" is probably more accurate, e.g.: `YOOOO CHATGEEEPEEETEEE!!!!!!1111 wasup I gots to asks you DIS.......`. Maybe someone can prove me wrong.

Re: How I write software with LLMs

#109

Post by lbreakjai »

akhrail1996 wrote: post #22

Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony…

The different models is a big one. In my workflow, I've got opus doing the deep thinking, and kimi doing the implementation. It helps manage costs.

Sample size of one, but I found it helps guard against the model drifting off. My different agents have different permissions. The worker can not edit the plan. The QA or planner can't modify the code. This is something I sometimes catch codex doing, modifying unrelated stuff while working.

Re: How I write software with LLMs

#110

Post by oytis »

I find the same problem applying to coding too. Even with everyone acting in good faith and reviewing everything themselves before pushing, you have essentially two reviwers instead of a writer and a reviewer, and there is no etiquette mandating how thoroughly the "author" should review their PR yet. It doesn't help if the amount of code to review gets larger (why would you go into agentic coding otherwise?)
Post reply on HN