AI coding and the peanut butter and jelly problem • Hacker News Board

Re: AI coding and the peanut butter and jelly problem

#81

Post by _wire_ » Sun, Apr 13, 2025, 10:59 PM UTC

Step 1: Computer, make me a peanut butter and jelly sandwich.

If this can't work, the program abstraction is insufficient to the task. This insufficiency is not a surprise.

That an ordinary 5-year can make a sandwich after only ever seeing someone make one, and that the sandwich so-made is a component within a life sustaining matrix which inevitably leads to new 5 year-olds making their own sandwiches and serenading the world about the joys of peanut butter and jelly is the crucial distinction between AI and intelligence.

The rest of the stuff about a Harvard professor ripping a hole in a bag and pouring jelly on a clump of bread on the floor is a kooky semantic game that reveals something about the limits of human intelligence among the academic elite.

We might wonder why some people have to get to university before encountering such basic epistemological conundrum as what constitutes clarity in exposition... But maybe that's what teaching to the test in U.S. K-12 gets you.

Alan Kay is known a riff on a simple study where Harvard students were asked what causes the earth's seasons: almost all of them give the wrong explanation, but many of them are very confident about the correctness of their wrong explanations.

Given that the measure of every AI chat program's performance is how agreeable its response is to a human, is there a clear distinction between a the human and the AI?

If this HN discussion was among AI chat programs considering their own situations and formulating understanding of their own problems; maybe waxing about the ineffable, for them, joy of eating a peanut butter and jelly sandwich...

But it isn't.

Re: AI coding and the peanut butter and jelly problem

#82

Post by daveguy » Mon, Apr 14, 2025, 12:40 AM UTC

kenjackson wrote: post #79

daveguy wrote: post #78

Earlier quoted context omitted.

I wasn't talking about how impressive AI systems are, or how far they've come. I was talking about the fact that any random human with any experience in a specific field -- even though they are not a domain expert -- is going to do better than an LLM. Or, human common sense >>>> what LLMs are doing.

We will have to agree to disagree about your fundamental point.

Fair enough. We will see.

Re: AI coding and the peanut butter and jelly problem

#83

Post by dsjoerg » Tue, Apr 15, 2025, 12:53 PM UTC

derefr wrote: post #67

dsjoerg wrote: post #57

Earlier quoted context omitted.

You're right about everything except you underestimate the current generation of LLMs. With the right prompting and guidance, they _already_ can give pushback and ask questions until satisfied.

Well, yes and no. You can in-context-learn an LLM into being a domain expert in a specific domain — at which point it'll start challenging you within that domain . But — AFAIK — you can't get current LLMs to do the thing that experienced programmers do, where they can "know you're wrong, even though they don't know why yet" — where the response isn't "no, that's wrong, and here's what's right:" but rather "I don't kn…

Yeah I agree that the current generation of LLMs dont appear to have been trained on solid "epistemological behavior". I believe the underlying architecture is capable of it, but I see signs of the training data not containing that sort of thing. In fact in either the training or the prompting or both it seems like the LLMs I use have been tuned to do the opposite.