Live data from Hacker News

An example of LLM prompting for programming

martinfowler.com

21–30 of 297 posts

Re: An example of LLM prompting for programming

#21

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

The problem is that it's not quite code. It's almost code, but without the precision, which puts it into a sort of Uncanny Valley of code-ness. It's detailed instructions for someone to write code, but the someone in this case is an alien or insane or on drugs so they might interpret it the way you meant it or they might go off on some weird tangent. You never know, and that means you'll need to check it with almost as much care as you'd take writing it.

Also, having it write its own tests doesn't mean those tests will themselves be correct let alone complete. This is a problem we already have with humans, because any blind spot they had while writing the code will still be present for writing the tests. Who hasn't found a bug in tests, leading to acceptance of broken code and/or rejection of correct alternatives? There's no reason to believe this problem won't also exist with an AI, and they have more blind spots to begin with.

Re: An example of LLM prompting for programming

#22

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

> LLMs can also write prompts and self introspect to debug.

Why should we assume that won't lead to a rabbit hole of misunderstanding or outright hallucination? If it doesn't know what "correct" really is, even infinite levels of supervision and reinforcement might still be toward an incorrect goal.

Re: An example of LLM prompting for programming

#23
For me chatGPT or phind (which is based on chatGPT4, if I understood right) are great documentation tools and also general productivity tools, nothing to say about it.

The main issue is that sometimes they really f** it up bad, they make you rethink your knowledge quite deeply (do I remember wrong? did I maybe understand this wrong? is chatGPT wrong?) and this is for me something that can be worse than having to do it myself, because it creates some sort of insecurity, as you always have to challenge your self thinking, and this is not how we work in our daily job, isn't it? At least this doesn't happen so frequently to me - from time to time we have arguments in the team, but this kind of "wrong information" feels more like "hidden" traps than someone else arguing (with valid arguments, of course).

Re: An example of LLM prompting for programming

#24

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

> by using English as spec, the barrier of entry has gone lower,

I'm not sure that is true. The level of back and forth and refinements needed indicate to me that the "English" used is not the normal language I use when talking to people.

It's almost like a refined version of cucumber with syntax that is slightly more forgiving.

Maybe I'm being a codger, but LLMs seem (at least for now) far better for summarizing and giving high level overviews of concepts rather than nailing precise code requirements.

Re: An example of LLM prompting for programming

#25

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

Except you now have a way “upwards” from an abstraction POV. Regular code is severely limited and highly surgical, by design. This is not. All these abstraction layers were invented to serve old style manual coders. Why bother explaining in great detail about “Konva” layers and react anymore? Give it a few years and let it finetune on IT tech and I see this being reduced to “I want whiteboard app with X general chara…

That "upwards" excludes a lot of relevant systems design logic that won't go away though, insofar as it is abstraction ad infinitum in the direction of fewer-relevant-details.

What'll happen is, details will continue to be relevant as tastes adjust to the new normal.

Like for my work, today, React is enterprise-ready, which is not good for me. It means it will likely dip my projects in unnecessary maintenance costs as compared to another widget of its type that does what I want in a lightweight manner. When I troubleshoot something of React's complexity, even my prompts will likely need to be longer.

But also, that's just one component of one component. And you have to experience this stuff in the first place, to know that you should pay attention to these details and not those other ones, for a given job, for a given client, in a given industry, with given specs.

So, if I was able to wave my hands I'd simply have all the problems I had back when I was a beginner. Ergo, it comes back to the clip art problem: Being able to buy clip art never made anyone a designer. But it made a lot of designers' jobs way easier.

We are simply regressing toward the mean with regard to programming. It was never about computers in the first place, never so concerned with syntax.

Anyway, back to browsing my theater program...

Re: An example of LLM prompting for programming

#26
post #9

I started a bit of an exploration around prompts and code a week or three back. I want to figure out the down/up-sides and create tools for myself around it. So, for this project (a game), I decided "for fun" to try to not write any code myself, and avoid narrow prompts that would just feed me single functions for a very specific purpose. The LLM should be responsible for this, not me! It's pretty painful since I sti…

Just curious, you’re using which version?

Re: An example of LLM prompting for programming

#27
post #9

I started a bit of an exploration around prompts and code a week or three back. I want to figure out the down/up-sides and create tools for myself around it. So, for this project (a game), I decided "for fun" to try to not write any code myself, and avoid narrow prompts that would just feed me single functions for a very specific purpose. The LLM should be responsible for this, not me! It's pretty painful since I sti…

That's a beautiful readme, starred!

Out of curiosity, right now would you say you have saved time by (almost) exclusively prompting instead of typing the code up yourself? Do you see that trending in another direction as the project progresses?

Re: An example of LLM prompting for programming

#28

I guess this is neat but I’d rather write code myself.

I'd rather farm all my own food, build my own house, and teach my own kids, but I don't have infinite time each day.

The prompt was almost as much work as the code, and there was no way to write that prompt without a CS education and/or years of development experience.

Re: An example of LLM prompting for programming

#29

If somebody thinks an LLM is coming for everybody's coding job, I'd say this article is a great counterpoint just for existing. You could tell someone from decades ago that we now use a very high level language for complex tasks in complex code ecosystems, never even mention AI, explain that the parser is really generalist-biased, and this article would make perfect sense as an example of exemplary code by a modern c…

The opposite might be true, and here’s why - 1) by using English as spec, the barrier of entry has gone lower, 2) LLMs can also write prompts and self introspect to debug.

I think English as a spec actually makes the barrier of entry higher, not lower. Code itself is far easier to understand than an English description of the code.

To understand an English description of code you already have to have a deeper understanding of what the code is doing. For code itself you can reference the syntax to understand what's going on.

The prompt in this case is using very technical language that a beginner will have no idea about. But if you gave them the code they could at least struggle along and figure it out by looking things up.

Re: An example of LLM prompting for programming

#30

I guess this is neat but I’d rather write code myself.

I'd rather farm all my own food, build my own house, and teach my own kids, but I don't have infinite time each day.

If you are optimizing for an end goal rather than enjoying the process... What is that goal? Why does it matter?
Post reply on HN