Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models do not recognize identifier swaps in Python

arxiv.org

Post reply on HN
Print view View original

41–50 of 103 posts

Re: Large language models do not recognize identifier swaps in Python

#41

Post by TeMPOraL »

mjburgess wrote: post #37
TeMPOraL wrote: post #28

Earlier quoted context omitted.

Yup. The whole premise of this test is idiotic. When you give a piece of code to an LLM, it doesn't magically switch into "code focused mode". They write: > Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers. But this is bullshit . Identifier renaming may be a no-op…

So the dialectic here is: > AI Industry: Generative AI is sensitive to the semantic properties of code > Credulous Fanatic: Yes! Of course! Here's an infinite number of cases to confirm that idea > Scientist: Here's a single case which shows that's false > Credulous Fanatic: But... "some irrelevant unevidenced point about human capabilities, 101 distractions, repetition of the latest OpenAI press release" Conclusion:…

No. The problem here is, "scientists" believe GPT models couldn't possibly be in any way exhibiting forms of thinking or intelligence similar to humans, and so they assume they must be doing some specific kind of computation humans are not, despite every interaction with ChatGPT being evidence to the contrary. They design a test that a human would not pass (despite their claims), but their imagined specific computational system would - and then act all proud and mighty after ChatGPT fails it too.

The only real insight from this event is: the authors of this paper imagined ChatGPT to be something it isn't, then demonstrated it in fact isn't it, and think they've discovered something.

Re: Large language models do not recognize identifier swaps in Python

#42

Post by mjburgess »

8organicbits wrote: post #14
extr wrote: post #4

Am I misunderstanding? The paper says: > We also carry out manual experiments on OpenAI ChatGPT-3.512 and GPT-4 models, where we interact with the models in multiple rounds of dialogue, trying to hint the correct solution. The models are still unable to provide the correct continuations. But if you look at the Appendix and the dialogues for GPT-3.5 and GPT-4, in the final turn of the GPT-4 program, it DOES (finally,…

> but better if made to continuously reflect With GPT-3.5 I've seen that asking the same question just gets different wrong answers. If one of them is eventually right, then the user needs to either know the right answer or be able to validate the answer. After a correct answer, if you ask again, you get more wrong answers. Still, amazing tech, but there's a big usability gap around correctness.

> there's a big usability gap around correctness

From an engineer's perspective I can see this seems minor -- but from a scientific perspective, it's kinda a crazy statement, right?

Imagine an alien who speaks english giving apparently correct descriptions of, eg., a room; and then, seemingly at random, saying wholly false things with the confidence (etc.) of its other statements.

A scientist studying this alien would therefore reasonably conclude that the alien wasn't at all sensitive to what the words mean, describe (etc.) but was engaged in a strange game of regurgitation which fools us much of the time.

ie., However it generates a response to a prompt, this "correctness issue" refutes the claim that it does so via being responsive to the meaning of the prompt.

I'm always quite disappointed by the extraordinary levels of credulity around this technology, and how it seems to take place --- in even somewhat scientific minds --- outside the bounds of ordinary science.

The relevant people to assess how a machine works, we're led to believe, is the credulous fools who use it. AI systems are valid targets of scientific analysis, as the above alien -- but the conclusions of this analysis do not fit the narrative.

Re: Large language models do not recognize identifier swaps in Python

#43

Post by concordDance »

mjburgess wrote: post #42
8organicbits wrote: post #14

Earlier quoted context omitted.

> but better if made to continuously reflect With GPT-3.5 I've seen that asking the same question just gets different wrong answers. If one of them is eventually right, then the user needs to either know the right answer or be able to validate the answer. After a correct answer, if you ask again, you get more wrong answers. Still, amazing tech, but there's a big usability gap around correctness.

> there's a big usability gap around correctness From an engineer's perspective I can see this seems minor -- but from a scientific perspective, it's kinda a crazy statement, right? Imagine an alien who speaks english giving apparently correct descriptions of, eg., a room; and then, seemingly at random, saying wholly false things with the confidence (etc.) of its other statements. A scientist studying this alien woul…

> A scientist studying this alien would therefore reasonably conclude that the alien wasn't at all sensitive to what the words mean, describe (etc.) but was engaged in a strange game of regurgitation which fools us much of the time.

This describes an unfortunately large fraction of humans too in my experience. :/

This is one of the reasons I'm impressed with LLMs, because they seem to fall into the same pitfalls less intelligent humans do.

Re: Large language models do not recognize identifier swaps in Python

#44

Post by pwdisswordfishc »

TeMPOraL wrote: post #28
thriftwy wrote: post #7

Human programmers will also initially spend 10x of effort working on code base where identifier names are meanongful but swapped. Doing so will employ a lot of inner dialogue such as "this method says close but it is actually reset" and I don't doubt LLMs may be made straight by the same prompts.

Yup. The whole premise of this test is idiotic. When you give a piece of code to an LLM, it doesn't magically switch into "code focused mode". They write: > Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers. But this is bullshit . Identifier renaming may be a no-op…

If a human being trips up over this 50% of the time (and I expect the actual figure to be lower), while an LLM trips up 90% of the time, that’s still a win for the human being.

Re: Large language models do not recognize identifier swaps in Python

#45

Post by fiso64 »

mjburgess wrote: post #36
twright0 wrote: post #20

Earlier quoted context omitted.

Sometimes! My attempt with GPT-4 yields a response where it acknowledges the print/len swap, but does not produce correct code in the end - it sort of loses track of what the original goal was. https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5a...

And if we're doing Science , ie., trying to explain how ChatGPT works and what it's intrinsic properties are --- this case is far more significant than the other. Inasmuch as the hypothesis that ChatGPT works "so as to be actually sensitive to the meaning of the code" is here falsified -- by a single case. An infinite number of apparent confirmations of this hypothesis are now Invalid !

Humans make foolish, nonsensical mistakes too. Does that mean that they're not intelligent and not sensitive to the meaning of the code?

Re: Large language models do not recognize identifier swaps in Python

#46

Post by golol »

TeMPOraL wrote: post #40
macleginn wrote: post #31

When reframed as a game, ChatGPT seems to understand the rules but then starts gaslighting me: Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. A: Certainly! In this game, the `print` function is swapped with the `len` function. Here's how yo…

Taken your prompt and slightly modified it: Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. Remember to take your time and think carefully, step by step. Solved with the following, on second try: A: To write a function using the swapped func…

If you prompt step by step thinking but the model writes the explanation after the solution, then you have not actually had the benefit of step nx step thinking. The model has to generate the explanation first so that the solution is generated with the information of an explanation first.

Re: Large language models do not recognize identifier swaps in Python

#47

Post by TeMPOraL »

golol wrote: post #46
TeMPOraL wrote: post #40

Earlier quoted context omitted.

Taken your prompt and slightly modified it: Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. Remember to take your time and think carefully, step by step. Solved with the following, on second try: A: To write a function using the swapped func…

If you prompt step by step thinking but the model writes the explanation after the solution, then you have not actually had the benefit of step nx step thinking. The model has to generate the explanation first so that the solution is generated with the information of an explanation first.

Agreed. However, I wasn't trying to do prompt engineering here - just warn it that it's a tricky case, as if I was dealing with a candidate for a junior software role on the interview (however stupid this kind of test would be on the interview).

Re: Large language models do not recognize identifier swaps in Python

#48

Post by starbugs »

concordDance wrote: post #43
mjburgess wrote: post #42

Earlier quoted context omitted.

> there's a big usability gap around correctness From an engineer's perspective I can see this seems minor -- but from a scientific perspective, it's kinda a crazy statement, right? Imagine an alien who speaks english giving apparently correct descriptions of, eg., a room; and then, seemingly at random, saying wholly false things with the confidence (etc.) of its other statements. A scientist studying this alien woul…

> A scientist studying this alien would therefore reasonably conclude that the alien wasn't at all sensitive to what the words mean, describe (etc.) but was engaged in a strange game of regurgitation which fools us much of the time. This describes an unfortunately large fraction of humans too in my experience. :/ This is one of the reasons I'm impressed with LLMs, because they seem to fall into the same pitfalls less…

Maybe it's just me, but I've yet to find a human who will confidently make up a function name that will do what I ask for and claim it is part of a standard library while that function does not exist at all. This is politician level stuff. It either means someone or something is trying to deceive in some way or doesn't have a complete understanding of the subject.

Re: Large language models do not recognize identifier swaps in Python

#49

Post by xcv123 »

mjburgess wrote: post #36
twright0 wrote: post #20

Earlier quoted context omitted.

Sometimes! My attempt with GPT-4 yields a response where it acknowledges the print/len swap, but does not produce correct code in the end - it sort of loses track of what the original goal was. https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5a...

And if we're doing Science , ie., trying to explain how ChatGPT works and what it's intrinsic properties are --- this case is far more significant than the other. Inasmuch as the hypothesis that ChatGPT works "so as to be actually sensitive to the meaning of the code" is here falsified -- by a single case. An infinite number of apparent confirmations of this hypothesis are now Invalid !

It's not black and white with these probabilistic models. The same input generated two outputs. Both were "actually sensitive to the meaning of the code", to varying degrees. One got it exactly right, one made an error, but partly got it right.

Re: Large language models do not recognize identifier swaps in Python

#50

Post by famouswaffles »

brenns10 wrote: post #33
IshKebab wrote: post #11

Earlier quoted context omitted.

Yeah it feels like the issue is that the researchers expect the AI to be smart enough to solve nonsensical Python puzzles but not smart enough to figure out that they are nonsensical and do the sane thing instead. Having said that, I did try telling ChatGPT explicitly it is a puzzle and not normal Python code, and every time it explained that it was a trick because the builtins were swapped... and then gave the wrong…

I think the issue is just that, however silly or pointless the puzzle is, it's clearly understandable by any human who knows Python. You just use "len" any time you mean "print", and "print" any time you mean "len". And these large language models, despite being language models, simply cannot handle this concept.

Going to paste together snippets from comments on this thread.

This bit is from the paper.

> Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers.

This is bullshit lol. Identifier renaming may be a no-op for the code, or even for the compiler/interpreter, but it's absolutely a meaningful change for human programmers. If it weren't, we'd all be calling functions named f0001, f0002, f0003, ... etc. to save space. Like us, LLMs process many different associations at the same time. A function named "print" has a lot of associations that together reinforce the understanding that it will make something the output of the program. Renaming it into len() makes the meaning go against all the association that go with that word.

Swapping print() and len() to use in the same program? That would trip any human up, it's tailor-made to be difficult for us to process. And, as it again turns out, so it is for LLMs, because they really seem to process associations the same way we do (at the gut/intuition/first reaction level).

Language Models can absolutely handle the concept. Proof is all over this thread. They just handle it very similarly to humans.

The problem here is, "scientists" believe GPT models couldn't possibly be in any way exhibiting forms of thinking or intelligence similar to humans, and so they assume they must be doing some specific kind of computation humans are not, despite every interaction with ChatGPT being evidence to the contrary. They design a test that a human would not pass (despite their claims), but their imagined specific computational system would - and then act all proud and mighty after ChatGPT fails it too. The only real insight from this event is: the authors of this paper imagined ChatGPT to be something it isn't, then demonstrated it in fact isn't it, and think they've discovered something.

Post reply on HN