Live data from Hacker News

An example of LLM prompting for programming

martinfowler.com

191–200 of 297 posts

Re: An example of LLM prompting for programming

#191

Earlier quoted context omitted.

Either way, you’re sending your companys biggest asset to another company, aren’t you? I’ll try these tools when they start being able to run locally

I sort of disagree that code is the biggest asset. Take the Yandex leak. What can you do with it? Outcompete them?

> Take the Yandex leak. What can you do with it?

Obviously, add it to the big training set of the next code model.

Re: An example of LLM prompting for programming

#192

Earlier quoted context omitted.

It's great to see that there's now a term for the type of prompting, “generated knowledge”. I've been experimenting with this technique since the beginning, and I've noticed a significant improvement in version 4. The process involves outlining the project, creating tasks, and feeding them back to chatGPT as you progress. This approach has helped me complete projects that would have otherwise taken me much longer to…

Something must be wrong with me. I could never get anything useful from Martin Fowler's writings, and coincidentally I cannot get any functional code out of ChatGPT. Even the boilerplate it produces for me needs to be corrected. I still use chatGPT to produce examples of abstract things but was not able to get any working code that matches concrete problems or even compiles.

Are you using the GPT4 model? There's a very significant improvement between 3.5 (the free one) and 4.

Re: An example of LLM prompting for programming

#193
post #183

Earlier quoted context omitted.

You iterate on your plan after it is generated step by step. You go and edit the prompt chain you started working on step 1 on, and modify it to start working on step 2 (including any ideas or fixes you have identified while implementing step 1. Repeat until complete. You can still absolutely hit the context limit, but you are far less likely to do so if you go back and start a new prompt chain for each different tho…

Great idea. But does it get hard to navigate back to something in older chat histories though? I find a new separate chat with the revised initial prompt to be easier.

I’ve been using another call to an LLM to write or rewrite code that is separate from the main “conversation”.

What I mean is that I’ve got a dialog going with an LLM and I’ve trained it to call a build() function with instructions that then returns the function, with the text of the function kept out of the dialog with the main thread.

Re: An example of LLM prompting for programming

#194
post #192

Earlier quoted context omitted.

Something must be wrong with me. I could never get anything useful from Martin Fowler's writings, and coincidentally I cannot get any functional code out of ChatGPT. Even the boilerplate it produces for me needs to be corrected. I still use chatGPT to produce examples of abstract things but was not able to get any working code that matches concrete problems or even compiles.

Are you using the GPT4 model? There's a very significant improvement between 3.5 (the free one) and 4.

I am supposedly on GPT4 via GPT+. I try using it for boilerplatey things, like terraform, and the results are simply incorrect. It seems more helpful in providing examples, even for some far more complex tech - like rust code.

Re: An example of LLM prompting for programming

#195

Earlier quoted context omitted.

I agree it's a good idea to take a moderate approach. The hype that LLMs are going to replace SWEs is clearly just that, hype, if you've done any real work trying to get GPT4 to give you the code you want. But it's also clearly a very useful tool. I think it'll absolutely destroy Stack Overflow.

> The hype that LLMs are going to replace SWEs is clearly just that, hype LLMs cannot replace anyone, but it is clear that engineers which master LLMs usage might multiply their productivity by a lot. The question is: If one LLM assisted engineer can work 10x faster, will companies reduce their engineer staff by 90%?

I've worked at far more companies with miles of product idea backlog we never get to than ones with nothing for engineering to do.

Now product will be able to use an LLM to come up with feature proposals and design docs even faster! :o

So: are you working at a company where engineering is a cost center or a revenue center? The latter wants to get more done at the same cost much more than it wants to just cut spend.

Re: An example of LLM prompting for programming

#196

Earlier quoted context omitted.

I agree it's a good idea to take a moderate approach. The hype that LLMs are going to replace SWEs is clearly just that, hype, if you've done any real work trying to get GPT4 to give you the code you want. But it's also clearly a very useful tool. I think it'll absolutely destroy Stack Overflow.

> The hype that LLMs are going to replace SWEs is clearly just that, hype LLMs cannot replace anyone, but it is clear that engineers which master LLMs usage might multiply their productivity by a lot. The question is: If one LLM assisted engineer can work 10x faster, will companies reduce their engineer staff by 90%?

To answer your question with a question if I may -- when did productivity increase in software ever result in headcount reduction? The competition also will have similar productivity gain.

Re: An example of LLM prompting for programming

#197
post #189

Earlier quoted context omitted.

> Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I can get similar results with Google sometimes and I can put together what I learned from different places. But I can get scripts that meet my exact requirements with ChatGPT. Most of my ChatGPT related code is scripting AWS related code an…

I think you (and kenjackson above) are misinterpreting what I was saying. I'm not saying use Google instead of ChatGPT; I'm saying pretend ChatGPT is Google and interact with the ChatGPT text prompt the same way. You don't need fully formed coherent sentences like you would when talking to a person; just drop in relevant keywords and ChatGPT will get you what you want.

Isn’t that the game changer though that you can use natural language and treat it like the “worlds smartest intern” and I can just give it the list of my requirements?

It’s the difference between:

“Python script to return all of the roles with a given policy AWS” (answer found on StackOverflow with Google)

And with ChatGPT

“Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list”

Re: An example of LLM prompting for programming

#198
post #79

Earlier quoted context omitted.

Artists' jobs are safe in part because they can also use AI, and most already use relevant ecosystems that now incorporate AI. Consumers who can operate AI for clip art purposes are simply still part of the same non-artist-paying demographic they always were. Same with code

As farmers' jobs were safe because farmers can use farming tools. These arguments don't track even vaguely. You are doing the equivalent of analyzing the future of solar power by assuming solar will cost the same in 10 years as it does today, and that each new watt of solar is matched 1:1 with new units of demand. Neither of these are sensible. It may be that ML code tools never displace many people, or even that the…

Industrialization has made farming tools incredibly complex, so I believe the statement "farmers' jobs were safe because farmers can use farming tools" is correct. You still need a farmer to farm, but you now need less manpower to farm. The specialist is secure while the untrained laborer is at risk.

Re: An example of LLM prompting for programming

#199
post #178

The article shows everything that works for this approach. But it's a bit disingenuous. At the end: > Once this is working, Xu Hao can repeat the process for the rest of the tasks in the master plan. No, he can't. After that much back and forth and getting it to fix little things where it gives responses with the full code listing again, he would have easily hit the token limit (at least with any chat LLM capable of…

Your experience matches mine closely. I've had ChatGPT-4 do great and then it just gets confused after a while. I can literally tell it "task X is done" and it'll apologise and show me a list of tasks where X is still not done - this is clearly not just a context window issue, as I have repeated variations of my statement over and over in the same session and the issue persists.

I have ended up using it the same way you have - it's honestly the best anti-procrastination tool I've ever used because I can tell it my intentions, what I've thought of so far... and it'll spit out a list of bite-sized chunks that get me going. I find myself looking forward to telling the AI I've completed a task.

Similarly, if I'm facing a tricky design decision, I find that just writing it out for ChatGPT is extremely helpful for clarifying my thought process. I actually used to do this conversational decision making process in a text editor long before ChatGPT, but when I know there's an AI on the other end my thinking becomes clearer and more goal-oriented. And unlike talking to myself or a human friend, it's happy to just say "well if these are your concerns, let's start HERE and then see what happens".

Re: An example of LLM prompting for programming

#200
post #189

Earlier quoted context omitted.

I think you (and kenjackson above) are misinterpreting what I was saying. I'm not saying use Google instead of ChatGPT; I'm saying pretend ChatGPT is Google and interact with the ChatGPT text prompt the same way. You don't need fully formed coherent sentences like you would when talking to a person; just drop in relevant keywords and ChatGPT will get you what you want.

Isn’t that the game changer though that you can use natural language and treat it like the “worlds smartest intern” and I can just give it the list of my requirements? It’s the difference between: “Python script to return all of the roles with a given policy AWS” (answer found on StackOverflow with Google) And with ChatGPT “Write a Python script that returns AWS IAM roles that contain one or more policies specified b…

> “Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list”

Again, this is completely unnecessary. This is like in the old days when technically illiterate people would quite literally Ask Jeeves[0] and search for full questions because they didn't know how to interface with a search engine.

A prompt that does exactly what you're asking: "python script get AWS IAM roles that contain a policy, policy as -p command line argument, output csv"

We'll see more of that terse, efficient, style as people get more comfortable, similar to how people have (mostly) stopped using full questions to search on Google. The "talk to ChatGPT like a human" part is entirely a distraction from taking advantage of the LLM for coding purposes. Perhaps more importantly, the responses being humanized is a distraction, too.

[0] https://en.wikipedia.org/wiki/Ask.com

Post reply on HN