Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An example of LLM prompting for programming

martinfowler.com

Post reply on HN
Print view View original

181–190 of 297 posts

Re: An example of LLM prompting for programming

#181

Post by Tostino »

afro88 wrote: post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

You iterate on your plan after it is generated step by step. You go and edit the prompt chain you started working on step 1 on, and modify it to start working on step 2 (including any ideas or fixes you have identified while implementing step 1. Repeat until complete.

You can still absolutely hit the context limit, but you are far less likely to do so if you go back and start a new prompt chain for each different thought process you are going through with it.

Re: An example of LLM prompting for programming

#182

Post by quijoteuniv »

afro88 wrote: post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

It's great to see that there's now a term for the type of prompting, “generated knowledge”. I've been experimenting with this technique since the beginning, and I've noticed a significant improvement in version 4. The process involves outlining the project, creating tasks, and feeding them back to chatGPT as you progress. This approach has helped me complete projects that would have otherwise taken me much longer to finish.

It's also useful for creating practical tutorials. While there are plenty of tutorials available online, sometimes you need guidance on a specific set of technologies. By using generated knowledge prompts, you can get a good outline and tasks to help you understand how these technologies interact.

One thing to keep in mind is to avoid derailing the conversation with questions that are not relevant to the core tasks. If you get stuck on something and need to debug, it's best to use a separate conversation to avoid derailing the project's progress and the allucinations & forgettingness

Re: An example of LLM prompting for programming

#183

Post by afro88 »

Tostino wrote: post #181
afro88 wrote: post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

You iterate on your plan after it is generated step by step. You go and edit the prompt chain you started working on step 1 on, and modify it to start working on step 2 (including any ideas or fixes you have identified while implementing step 1. Repeat until complete. You can still absolutely hit the context limit, but you are far less likely to do so if you go back and start a new prompt chain for each different tho…

Great idea. But does it get hard to navigate back to something in older chat histories though?

I find a new separate chat with the revised initial prompt to be easier.

Re: An example of LLM prompting for programming

#184

Post by afro88 »

quijoteuniv wrote: post #182
afro88 wrote: post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

It's great to see that there's now a term for the type of prompting, “generated knowledge”. I've been experimenting with this technique since the beginning, and I've noticed a significant improvement in version 4. The process involves outlining the project, creating tasks, and feeding them back to chatGPT as you progress. This approach has helped me complete projects that would have otherwise taken me much longer to…

Absolutely, and same here. I've done multiple tools that would have taken 2-3 days each in 2-3 hours each.

> One thing to keep in mind is to avoid derailing the conversation with questions that are not relevant to the core tasks. If you get stuck on something and need to debug, it's best to use a separate conversation to avoid derailing the project's progress and the allucinations & forgettingness

Definitely. Great advice.

Another tip: don't bother asking it to fix small things. Just mention you fixed it in the next reply and move on.

Re: An example of LLM prompting for programming

#185

Post by themodelplumber »

rootusrootus wrote: post #123
themodelplumber wrote: post #66

Earlier quoted context omitted.

Ni-dominant. It exists nowadays in various post-Jungian models, many of which are really fascinating, having fleshed it out a lot. The opposing function to Ni is Se, which creates a dichotomy of planning/foreseeing vs. doing/performing. The functions oscillate as a kind of duty cycle, so a lot of sages out there have hobbies as musicians, stage magicians, etc. This dichotomy also effectively shuts out detail memory f…

I would like to subscribe to your newsletter. Even if approximately 75% of that sailed right over my head.

Best I can do is RSS!

Re: An example of LLM prompting for programming

#186

Post by bartimus »

nextworddev wrote: post #18
themodelplumber wrote: post #8

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

But there's still going to have to be a human who has the ability to form a mental model of the thing that's needing to be implemented. Functionally and technically. The results of the LLM will vary depending on the level of know-how the human instructor has.

Re: An example of LLM prompting for programming

#187

Post by AzzieElbab »

quijoteuniv wrote: post #182
afro88 wrote: post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

It's great to see that there's now a term for the type of prompting, “generated knowledge”. I've been experimenting with this technique since the beginning, and I've noticed a significant improvement in version 4. The process involves outlining the project, creating tasks, and feeding them back to chatGPT as you progress. This approach has helped me complete projects that would have otherwise taken me much longer to…

Something must be wrong with me. I could never get anything useful from Martin Fowler's writings, and coincidentally I cannot get any functional code out of ChatGPT. Even the boilerplate it produces for me needs to be corrected. I still use chatGPT to produce examples of abstract things but was not able to get any working code that matches concrete problems or even compiles.

Re: An example of LLM prompting for programming

#188

Post by visarga »

yodsanklai wrote: post #50
mjr00 wrote: post #49

Earlier quoted context omitted.

Absolutely. People will quickly realize that for coding, the natural language part of LLMs is a distraction. Copilot is much better for someone actually writing code, but unfortunately doesn't get as emphasized due to the narrative surrounding LLMs right now.

> Copilot is much better for someone actually writing code I haven't used copilot yet, but I'm using occasionally chatgpt with prompts such as "write a bash/python script take takes these parameters and perform this tasks". Then I iterate if needed, and usually, i can get what i want faster than without using chatgpt. It's not a game changer, but it's a performance boost. How natural language is a distraction here? a…

> It's not a game changer, but it's a performance boost.

The story of all AI in 2023 - maybe 2x performance improvement, maybe a bit less. The big problem is that you can't trust it on its own, so it doesn't improve productivity 100x. Not even a receipt reader is good enough to reach 100%, you got to check the total, maybe it missed the dot and you get the 100x boost after all.

Re: An example of LLM prompting for programming

#189

Post by mjr00 »

scarface74 wrote: post #174
mjr00 wrote: post #56

Earlier quoted context omitted.

Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I just typed in "bash script recursive chmod 777 all files" (as a dumb toy example) and got a resulting script back. It was surrounded by two natural language GPT comments: > It's generally not recommended to give all files and directories the…

> Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I can get similar results with Google sometimes and I can put together what I learned from different places. But I can get scripts that meet my exact requirements with ChatGPT. Most of my ChatGPT related code is scripting AWS related code an…

I think you (and kenjackson above) are misinterpreting what I was saying. I'm not saying use Google instead of ChatGPT; I'm saying pretend ChatGPT is Google and interact with the ChatGPT text prompt the same way. You don't need fully formed coherent sentences like you would when talking to a person; just drop in relevant keywords and ChatGPT will get you what you want.

Re: An example of LLM prompting for programming

#190

Post by tarruda »

rootusrootus wrote: post #155
gdubs wrote: post #154

There's an unfortunately common take on AI that goes basically like this: "I tried it and it didn't do what I wanted, not impressed." My suggestion is to tune out the noise and really try experimenting with these tools – and know that they're rapidly improving. Even if ultimately you have criticisms or decide one way or another, at least really investigate them for your own use-cases rather than jumping on a bandwago…

I agree it's a good idea to take a moderate approach. The hype that LLMs are going to replace SWEs is clearly just that, hype, if you've done any real work trying to get GPT4 to give you the code you want. But it's also clearly a very useful tool. I think it'll absolutely destroy Stack Overflow.

> The hype that LLMs are going to replace SWEs is clearly just that, hype

LLMs cannot replace anyone, but it is clear that engineers which master LLMs usage might multiply their productivity by a lot.

The question is: If one LLM assisted engineer can work 10x faster, will companies reduce their engineer staff by 90%?

Post reply on HN