An example of LLM prompting for programming
martinfowler.com
An example of LLM prompting for programming
1–10 of 297 posts
Re: An example of LLM prompting for programming
#2I guess this is neat but I’d rather write code myself.
Re: An example of LLM prompting for programming
#3I guess this is neat but I’d rather write code myself.
[deleted]
Re: An example of LLM prompting for programming
#4The article stresses to never put anything that may be confidential into the prompt. Yet, chatGpt offers to out-out from using your data for training.
For most purposes that seems to be sufficient doesn't it? Or are there reasons not to trust OpenAi on this one?
Re: An example of LLM prompting for programming
#5The article stresses to never put anything that may be confidential into the prompt . Yet, chatGpt offers to out-out from using your data for training. For most purposes that seems to be sufficient doesn't it? Or are there reasons not to trust OpenAi on this one?
> Or are there reasons not to trust OpenAi on this one?
Yes, more related to general tech history and not a dig on OpenAI though.
Re: An example of LLM prompting for programming
#6The article stresses to never put anything that may be confidential into the prompt . Yet, chatGpt offers to out-out from using your data for training. For most purposes that seems to be sufficient doesn't it? Or are there reasons not to trust OpenAi on this one?
From a GDPR or commercial confidentiality perspective, it doesn't matter what OpenAI say they'll do with your data, you can't share it with them.
Let's say your doctor enters sensitive info about you, and despite having told OpenAI not to train data with it, they use it anyway due to a bug. A year from now, ChatGPT is generating personal information tells everyone and anyone about your sensitive info.
Would you exclusively blame ChatGPT?
Re: An example of LLM prompting for programming
#7The article stresses to never put anything that may be confidential into the prompt . Yet, chatGpt offers to out-out from using your data for training. For most purposes that seems to be sufficient doesn't it? Or are there reasons not to trust OpenAi on this one?
There was a bug where chat history of some users were visible to others
Re: An example of LLM prompting for programming
#8If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing.
You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern coder working for a living.
That's code in there, the stuff Xu Hao is writing.
And also, that's not even getting into the debugging part... Which will be about other code, that looks different.
Re: An example of LLM prompting for programming
#9I started a bit of an exploration around prompts and code a week or three back. I want to figure out the down/up-sides and create tools for myself around it.
So, for this project (a game), I decided "for fun" to try to not write any code myself, and avoid narrow prompts that would just feed me single functions for a very specific purpose. The LLM should be responsible for this, not me! It's pretty painful since I still have to debug and understand the potential garbage I was given and after understanding what is wrong, get rid of it, and change/add to the prompt to get new code. Very often completely new code[1]. Rinse and repeat until I have what I need.
The above is a contrived scenario, but it does give some interesting insights. A nice one is that since here is one or more prompts connected to all the code (and its commit), the intention of the code is very well documented in natural language. The commit history creates a rather nice story that I would not normally get in a repository.
Another thing is, getting an LLM (ChatGPT mostly) to fix a bug is really hit and miss and mostly miss for me. Say, a buggy piece comes from the LLM and I feel that this could almost be what I need. I feed that back in with a hint or two and it's very rare that it actually fixes something unless I am very very specific (again, needing to read/understand the intention of the solution). In many cases I, again, get completely new code back. This, more than once, forced my hand to "cheat" and do human changes or additions.
Due to the nature of the contrived scenario, the code quality is obviously suffering but I am looking forward to making the LLM refactor/clean things up eventually.
On occasion ChatGPT tells me it can't help me with my homework. Which is interesting in itself. They are actually trying (but failing) to prevent that. I am really curious how gimped their models will be going forward.
I've been programming for quite long. I've come to realize that I don't need to be programming in the traditional sense. What I like is creating. If that means I can massage an LLM to do a bit of grunt work, I'm good with that.
That said, it still often feels very much like programming, though.
[1] The completely new code issue can likely be alleviated by tweaking transformers settings
Edit: For the curious, the repo is here: https://github.com/romland/llemmings and an example of a commit from the other day: https://github.com/romland/llemmings/commit/466babf420f617dd... - I will push through and make it a playable game, after that, I'll see.
Re: An example of LLM prompting for programming
#10Is there a tool that allows to do this within a text editor (for instance VS Code). Using selection instead of copy pasting. Having the LLM store its output directly within local files. Maybe giving it access to a shell to run the tests on its own ?