AI coding and the peanut butter and jelly problem • Hacker News Board

Re: AI coding and the peanut butter and jelly problem

#61

Post by kenjackson » Sat, Apr 12, 2025, 12:58 PM UTC

barotalomey wrote: post #53

kenjackson wrote: post #18

This is actually no different than for humans once you get past the familiar. It's like the famous project management tree story: https://pmac-agpc.ca/project-management-tree-swing-story If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities. But they are MUCH worse than humans when it comes to creating images/videos.

> If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities. That's demonstrateably false, as proven by both OpenAI's own research [1] and endless independent studies by now. What is fascinating is how some people cling on false ideas about what LLM is and isnt. Its a recurring fallacy that's bound to get it's own name any time soon. 1: https://news.y…

You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote.

Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose?

Re: AI coding and the peanut butter and jelly problem

#62

Post by aleph_minus_one » Sat, Apr 12, 2025, 1:18 PM UTC

kenjackson wrote: post #61

barotalomey wrote: post #53

Earlier quoted context omitted.

> If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities. That's demonstrateably false, as proven by both OpenAI's own research [1] and endless independent studies by now. What is fascinating is how some people cling on false ideas about what LLM is and isnt. Its a recurring fallacy that's bound to get it's own name any time soon. 1: https://news.y…

You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote. Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US)…

> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose?

You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.

Re: AI coding and the peanut butter and jelly problem

#63

Post by kenjackson » Sat, Apr 12, 2025, 1:46 PM UTC

aleph_minus_one wrote: post #62

kenjackson wrote: post #61

Earlier quoted context omitted.

You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote. Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US)…

> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose? You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.

Brain surgery requires a license.

But I do expect an arbitrary adult to be able to follow instructions.

Ok. How about you give me a text based task where you would pick the random adult over the LLM?

Re: AI coding and the peanut butter and jelly problem

#64

Post by MBCook » Sat, Apr 12, 2025, 2:08 PM UTC

LikesPwsh wrote: post #54

MBCook wrote: post #30

Earlier quoted context omitted.

Exactly. This same point was mentioned on Accidental Tech Podcast last week during a section primarily about “vibe coding”. (May have been the paid-only segment) If the LLM gets something wrong, you have to be more exact to get it to make the program do the thing you want. And when that isn’t perfect, you have to tell it exactly what you want to to do in THAT situation. And the next one. And the next one. At that poi…

I realise this is meant to be a jab at high-level programming languages, but SQL really did succeed at that. Its abstraction may leak sometimes, but most people using it are incredibly productive without needing to learn what a spool operator or bitmap does. Even though the GUI and natural language aspects of 4GL failed, declarative programming was worth it.

I really like SQL personally. You’re right it does work well, but I suspect that’s because it has a limited domain instead of being a general purpose language.

Re: AI coding and the peanut butter and jelly problem

#65

Post by aleph_minus_one » Sat, Apr 12, 2025, 2:12 PM UTC

kenjackson wrote: post #63

aleph_minus_one wrote: post #62

Earlier quoted context omitted.

> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose? You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.

Brain surgery requires a license. But I do expect an arbitrary adult to be able to follow instructions. Ok. How about you give me a text based task where you would pick the random adult over the LLM?

> Brain surgery requires a license.

This is rather a red-tape problem. :-)

Re: AI coding and the peanut butter and jelly problem

#66

Post by grahac » Sat, Apr 12, 2025, 2:18 PM UTC

daxfohl wrote: post #39

MBCook wrote: post #30

Earlier quoted context omitted.

Exactly. This same point was mentioned on Accidental Tech Podcast last week during a section primarily about “vibe coding”. (May have been the paid-only segment) If the LLM gets something wrong, you have to be more exact to get it to make the program do the thing you want. And when that isn’t perfect, you have to tell it exactly what you want to to do in THAT situation. And the next one. And the next one. At that poi…

This is what I keep coming back to. I'm sure I'm not the only one here who frequently writes the code, or at least a PoC, then writes the design doc based on it. Because the code is the most concise and precise way to specify what you really want. And writing it gives you more clarity on things you might not have thought about when writing it in a document. Unrolling that into pseudocode/English almost always gets co…

Agreed although AIs today with simple project based rules can do things like check and account for error cases, and write the appropriate unit tests for those error cases.

I personally have found I can often create equivalent code in less English than typing.

Also it works very well where the scope is well defined like implementing interfaces or porting a library from one language to another.

Re: AI coding and the peanut butter and jelly problem

#67

Post by derefr » Sat, Apr 12, 2025, 3:05 PM UTC

dsjoerg wrote: post #57

derefr wrote: post #38

> Today’s AI Still Has a PB&J Problem If this is how you're modelling the problem, then I don't think you learned the right lesson from the PB&J "parable." Here's a timeless bit of wisdom, several decades old at this point: Managers think that if you can just replace code with something else that isn't text with formal syntax , then all the sudden "regular people" (like them, maybe?) will be able to "program" a syste…

You're right about everything except you underestimate the current generation of LLMs. With the right prompting and guidance, they _already_ can give pushback and ask questions until satisfied.

Well, yes and no.

You can in-context-learn an LLM into being a domain expert in a specific domain — at which point it'll start challenging you within that domain.

But — AFAIK — you can't get current LLMs to do the thing that experienced programmers do, where they can "know you're wrong, even though they don't know why yet" — where the response isn't "no, that's wrong, and here's what's right:" but rather "I don't know about that... one minute, let me check something" — followed by motivated googling, consulting docs, etc.

And yes, the "motivated googling" part is something current models (DeepResearch) are capable of. But the layer above that is missing. You need a model with:

1. trained-in reflective awareness — "knowing what you know [and what you don't]" — such that there's a constant signal within the model representing "how confident I am in the knowledge / sources that I'm basing what I'm saying upon", discriminated as a synthesis/reduction over the set of "memories" the model is relying upon;

2. and a trained-in capability to evaluate the seeming authoritativeness and domain experience of the user, through their statements (or assertions-from-god in the system prompt about the user) — in order for the model to decide whether to trust a statement you think sounds "surprising", vs. when to say "uhhhhh lemme check that."

Re: AI coding and the peanut butter and jelly problem

#68

Post by derefr » Sat, Apr 12, 2025, 3:09 PM UTC

taurath wrote: post #55

derefr wrote: post #41

Earlier quoted context omitted.

An LLM is a very effective human-solution-description / pseudocode to "the ten programming languages we use at work, where I'm only really fluent in three of them, and have to use language references for the others each time I code in them" transpiler. It also remembers CLI tool args far better than I do. Before LLMs, I would often have to sit and just read a manpage in its entirety to see if a certain command-line t…

I’ve found that the surfacing of tools and APIs really can help me dive into learning, but ironically usually by AI finding a tool and then me reading its documentation, as I want to understand if it has the capabilities or flexibility I have in mind. I can leave that to LLMs to tell me, but I find it’s too good an opportunity to build my own internal knowledge base to pass up. It’s the back and forth between having…

Exactly — one thing LLMs are great at, is basically acting as a coworker who happens to have a very wide breadth of knowledge (i.e. to know at least a little about a lot) — who you can thus ask to "point you in a direction" any time you're stuck or don't know where to start.

Re: AI coding and the peanut butter and jelly problem

#69

Post by aleph_minus_one » Sat, Apr 12, 2025, 3:14 PM UTC

MBCook wrote: post #30

zahlman wrote: post #6

Okay, but like. If you do have that skill to communicate clearly and describe the requirements of a novel problem, why is the AI still useful? Actually writing the code should be relatively trivial from there. If it isn't, that points to a problem with your tools/architecture/etc. Programmers IMX are, on average, far too tolerant of boilerplate.

Exactly. This same point was mentioned on Accidental Tech Podcast last week during a section primarily about “vibe coding”. (May have been the paid-only segment) If the LLM gets something wrong, you have to be more exact to get it to make the program do the thing you want. And when that isn’t perfect, you have to tell it exactly what you want to to do in THAT situation. And the next one. And the next one. At that poi…

> At that point you’re programming. It may not be the same as coding in a traditional language, but isn’t it effectively the same process? You’re having to lay out all the exact steps to take when different things happen.

No, it isn't.

Programming is thinking deeply about

- the invariants that your code obeys

- which huge implications a small, innocent change in one part of the program will have for other, seemingly unrelated parts of the program

- in which sense the current architecture is (still) the best possible for what the program does, and if not, what the best route is to get there

- ...

Re: AI coding and the peanut butter and jelly problem

#70

Post by daveguy » Sat, Apr 12, 2025, 5:10 PM UTC

aleph_minus_one wrote: post #62

kenjackson wrote: post #61

Earlier quoted context omitted.

You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote. Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US)…

> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose? You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.

I would chose a random person from my company that was hired to work in that domain to solve problems in that domain. Yes, regardless of the position. Accountant in the domain, yes. Office organizer in the domain, yes. Essentially anyone in the domain, yes. No offense, but by restricting the selection to the general human population you're setting a low bar for LLMs here.