Live data from Hacker News

AI coding and the peanut butter and jelly problem

iamcharliegraham.substack.com

41–50 of 83 posts

Re: AI coding and the peanut butter and jelly problem

#41
post #6

Okay, but like. If you do have that skill to communicate clearly and describe the requirements of a novel problem, why is the AI still useful? Actually writing the code should be relatively trivial from there. If it isn't, that points to a problem with your tools/architecture/etc. Programmers IMX are, on average, far too tolerant of boilerplate.

An LLM is a very effective human-solution-description / pseudocode to "the ten programming languages we use at work, where I'm only really fluent in three of them, and have to use language references for the others each time I code in them" transpiler.

It also remembers CLI tool args far better than I do. Before LLMs, I would often have to sit and just read a manpage in its entirety to see if a certain command-line tool could do a certain thing. (For example: do you know off-hand if you can get ls(1) to format file mtimes as ISO8601 or POSIX timestamps? Or — do you know how to make find(1) prune a specific subdirectory, so that it doesn't have to iterate-over-and-ignore the millions of tiny files inside it?) But now, I just ask the LLM for the flags that will make the tool do the thing; it spits them out (if they exist); and then I can go and look at the manpage and jump directly to that flag to learn about it — using the manpage as a reference, the way it was intended.

Actually, speaking of CLI tools, it also just knows about tools that I don't. You have to be very good with your google-fu to go from the mental question of "how do I get disk IO queue saturation metrics in Linux?" to learning about e.g. the sar(1) command. Or you can just ask an LLM that actual literal question.

Re: AI coding and the peanut butter and jelly problem

#42
Whenever I'm prompting LLM for this kind of thing I find myself wishing there was a BASIC style protocol that we could use to instruct LLMs. Numbered statements, GOTOs to jump around, standardized like MCP or A2A such that all LLMs are trained to understand and verified to follow the logic.

Why BASIC? It's a lot harder to mix English and structured programming concepts. Plus it's nice if you forget a step between 20 and 30 you can just say `25 print 'halfway'` from the chat.

Re: AI coding and the peanut butter and jelly problem

#43
post #20
post #8

Earlier quoted context omitted.

Once you've got to a detailed specification, LLMs are a lot faster at correctly typing code than you are.

As a developer, typing speed is rarely the bottleneck.

Old trope that is no longer true.

Re: AI coding and the peanut butter and jelly problem

#44
post #42

Whenever I'm prompting LLM for this kind of thing I find myself wishing there was a BASIC style protocol that we could use to instruct LLMs. Numbered statements, GOTOs to jump around, standardized like MCP or A2A such that all LLMs are trained to understand and verified to follow the logic. Why BASIC? It's a lot harder to mix English and structured programming concepts. Plus it's nice if you forget a step between 20…

I've had successful chats with ChatGPT that uses Python as pseudocode.

I specifically mean pseudocode intended to clarify and communicate - as opposed to be run by a computer.

E.g.: https://chatgpt.com/share/67f9fee9-07dc-8003-a272-ca05d91282...

Re: AI coding and the peanut butter and jelly problem

#45
post #37

lol, I didn't realize how famous the PB&J exercise was. That's fantastic. I thought it was just from this puppet video I've been showing my 4yo and his friends. Anyway they seem to love it. https://m.youtube.com/watch?v=RmbFJq2jADY&t=3m25s Also seems like great advice, feels like a good description of what Ive been gravitating towards / having more luck with lately proompting.

My class did it with paper airplanes. My partner used the phrase "hotdog style" which I'd never heard. Good times!

Re: AI coding and the peanut butter and jelly problem

#47
My teacher did the peanut butter and jelly problem with us in the fourth grade, but we were given the time to write the instructions as homework and she picked a few to execute the following day.

The disappointment always stayed with me that my instructions were not chosen, as I really had been far more precise than the fun examples she did choose. I recall even explaining which side of the knife to use when taking peanut butter from the jar.

Of course, she would still have found plenty of bugs in my instructions, which I wish I still had.

Thanks for that, and also the pet rats, Ms. Clouser!

Re: AI coding and the peanut butter and jelly problem

#48
post #39
post #30

Earlier quoted context omitted.

Exactly. This same point was mentioned on Accidental Tech Podcast last week during a section primarily about “vibe coding”. (May have been the paid-only segment) If the LLM gets something wrong, you have to be more exact to get it to make the program do the thing you want. And when that isn’t perfect, you have to tell it exactly what you want to to do in THAT situation. And the next one. And the next one. At that poi…

This is what I keep coming back to. I'm sure I'm not the only one here who frequently writes the code, or at least a PoC, then writes the design doc based on it. Because the code is the most concise and precise way to specify what you really want. And writing it gives you more clarity on things you might not have thought about when writing it in a document. Unrolling that into pseudocode/English almost always gets co…

Another issue I see is the "Machine Stops" problem. When we come to depend on a systems that fails to foster the skills and knowledge needed to reproduce it (i.e. if programming comes to be so easy to so many people that they don't actually need to know how it works under the hood) you slowly loose the ability to maintain and extend the system as a society.

Re: AI coding and the peanut butter and jelly problem

#49
post #34
post #32

Earlier quoted context omitted.

The point of the PB&J thing is exactly to demonstrate that your native tongue isn’t precise enough to program a computer with. There’s a reason those interfaces are rigid, and it’s not “gatekeeping”. (The cryptic part is just to increase information density — see COBOL for an alternative.)

I think https://docs.cursor.com/chat/agent has shown plain English is precise enough to program a computer with, and some well respected programmers have become fans of it https://x.com/karpathy/status/1886192184808149383 I only took exception to the original statement - that coding is trivial, and the questioning if AI is even useful. So many people are finally able to create things they were never able to. That's s…

> intelligent beings can communicate in the context of a lot of prior knowledge

This is key. It works because of previous work. People have shared context because they develop it over time, when we are raised - shared context is passed on the the new generation and it grows.

LLMs consume the context recorded in the training data, but they don't give it back. They diminish it because people don't need to learn the shared context when using this tools. It appears to work in some use cases, but it will degrade our collective shared context over time as people engage with and use these tools that consume past shared context and at the same time atrophy our ability to maintain and increase the shared context. Because the shared context is reproduced and grows when it is learned by people. If a tool just takes it and precludes people learning it, there is a delayed effect where over time there will be less shared context and when the performance of the tool degrades the ability to maintain and extend the shared context will also have degraded. We might get to an irrecoverable state and spiral.

Re: AI coding and the peanut butter and jelly problem

#50

This is actually no different than for humans once you get past the familiar. It's like the famous project management tree story: https://pmac-agpc.ca/project-management-tree-swing-story If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities. But they are MUCH worse than humans when it comes to creating images/videos.

[deleted]
Post reply on HN