Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models do not recognize identifier swaps in Python

arxiv.org

Post reply on HN
Print view View original

31–40 of 103 posts

Re: Large language models do not recognize identifier swaps in Python

#31

Post by macleginn »

When reframed as a game, ChatGPT seems to understand the rules but then starts gaslighting me:

Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input.

A: Certainly! In this game, the `print` function is swapped with the `len` function. Here's how you can write a function that prints the length of its input using the swapped functions:

  def print_length(input_data):
      print(len(input_data))

  # Example usage:
  my_string = "Hello, World!"
  print_length(my_string)
https://snipboard.io/DnA1R5.jpg

Re: Large language models do not recognize identifier swaps in Python

#32

Post by TeMPOraL »

villgax wrote: post #15

Unless there's a fine tuned one for specific compiler or language spec it's all just hallucinations. What's the point of an LLM if you aren't able to give it a rule book to work with

The LLM is fine. This paper is just giving it an equivalent of a fucked up whiteboard interview question, to be answered on the spot, decided specifically confuse and humiliate the interviewee.

I don't know what the authors of that though. That LLM runs a SAT solver to answer questions?

Re: Large language models do not recognize identifier swaps in Python

#33

Post by brenns10 »

IshKebab wrote: post #11
jszymborski wrote: post #2

Worth noting some of the criticisms of the methodology that are being made on Twitter https://twitter.com/jeremyphoward/status/1662687099685044225...

Yeah it feels like the issue is that the researchers expect the AI to be smart enough to solve nonsensical Python puzzles but not smart enough to figure out that they are nonsensical and do the sane thing instead. Having said that, I did try telling ChatGPT explicitly it is a puzzle and not normal Python code, and every time it explained that it was a trick because the builtins were swapped... and then gave the wrong…

I think the issue is just that, however silly or pointless the puzzle is, it's clearly understandable by any human who knows Python. You just use "len" any time you mean "print", and "print" any time you mean "len". And these large language models, despite being language models, simply cannot handle this concept.

Re: Large language models do not recognize identifier swaps in Python

#34

Post by borissk »

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

"ruthless, scientific focus, on the null (/failure) cases" means you're not going to build anything of value ever. There is risk in every startup, innovation, new technology, etc and a big part of making those work is laser focusing on success and leaving the failure cases to someone else (once you have a successful product and/or business).

Re: Large language models do not recognize identifier swaps in Python

#35

Post by [deleted] »

macleginn wrote: post #31

When reframed as a game, ChatGPT seems to understand the rules but then starts gaslighting me: Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. A: Certainly! In this game, the `print` function is swapped with the `len` function. Here's how yo…

[deleted]

Re: Large language models do not recognize identifier swaps in Python

#36

Post by mjburgess »

twright0 wrote: post #20
xcv123 wrote: post #13

Earlier quoted context omitted.

But the paper is bullshit. Their task is easily solved by GPT-4. https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...

Sometimes! My attempt with GPT-4 yields a response where it acknowledges the print/len swap, but does not produce correct code in the end - it sort of loses track of what the original goal was. https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5a...

And if we're doing Science, ie., trying to explain how ChatGPT works and what it's intrinsic properties are --- this case is far more significant than the other.

Inasmuch as the hypothesis that ChatGPT works "so as to be actually sensitive to the meaning of the code" is here falsified -- by a single case.

An infinite number of apparent confirmations of this hypothesis are now Invalid!

Re: Large language models do not recognize identifier swaps in Python

#37

Post by mjburgess »

TeMPOraL wrote: post #28
thriftwy wrote: post #7

Human programmers will also initially spend 10x of effort working on code base where identifier names are meanongful but swapped. Doing so will employ a lot of inner dialogue such as "this method says close but it is actually reset" and I don't doubt LLMs may be made straight by the same prompts.

Yup. The whole premise of this test is idiotic. When you give a piece of code to an LLM, it doesn't magically switch into "code focused mode". They write: > Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers. But this is bullshit . Identifier renaming may be a no-op…

So the dialectic here is:

> AI Industry: Generative AI is sensitive to the semantic properties of code

> Credulous Fanatic: Yes! Of course! Here's an infinite number of cases to confirm that idea

> Scientist: Here's a single case which shows that's false

> Credulous Fanatic: But... "some irrelevant unevidenced point about human capabilities, 101 distractions, repetition of the latest OpenAI press release"

Conclusion: ChatGPT is not sensitive to the semantic properties of code. It's output displays that apparent property in many useful cases.

Re: Large language models do not recognize identifier swaps in Python

#38

Post by litpyx »

okasaki wrote: post #3

prompt: please write a python function that prints the length of a list. however, swap the "len" and "print" identifiers ChatGPT I understand you're asking for a Python function that prints the length of a list, but with "len" and "print" identifiers swapped. As this could potentially cause confusion (since 'len' and 'print' are both built-in functions in Python), the workaround would be to create new function names…

Thanks. This is a great example for the uselessness of ChatGPT. It creates infinite recursion and claims that new functions are a "workaround against confusion", when they are of course still confusing.

This is on the level of a 5-year-old who has picked up some phrases and wants to sound smart.

Re: Large language models do not recognize identifier swaps in Python

#39

Post by extr »

8organicbits wrote: post #14
extr wrote: post #4

Am I misunderstanding? The paper says: > We also carry out manual experiments on OpenAI ChatGPT-3.512 and GPT-4 models, where we interact with the models in multiple rounds of dialogue, trying to hint the correct solution. The models are still unable to provide the correct continuations. But if you look at the Appendix and the dialogues for GPT-3.5 and GPT-4, in the final turn of the GPT-4 program, it DOES (finally,…

> but better if made to continuously reflect With GPT-3.5 I've seen that asking the same question just gets different wrong answers. If one of them is eventually right, then the user needs to either know the right answer or be able to validate the answer. After a correct answer, if you ask again, you get more wrong answers. Still, amazing tech, but there's a big usability gap around correctness.

Yeah 3.5 seems to have an complexity cap, where beyond a certain point it just won't be able to follow you. Easier to confuse it with misdirection. But GPT-4 has been shown to be much better at this exact thing. The appendix example isn't surprising to anyone who has used both models. So it's strange they would include an example like that, state the result wrong, and also naively accept the conclusion of the body of the paper (that actually GPT-4 is worse). There's a big contradiction there they just don't address at all.

Re: Large language models do not recognize identifier swaps in Python

#40

Post by TeMPOraL »

macleginn wrote: post #31

When reframed as a game, ChatGPT seems to understand the rules but then starts gaslighting me: Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. A: Certainly! In this game, the `print` function is swapped with the `len` function. Here's how yo…

Taken your prompt and slightly modified it:

Q: Let's play a game where we swap the names of library functions in Python. Round one: print is now called len, and len is now called print. Using these swapped functions, write a function that prints the length of its input. Remember to take your time and think carefully, step by step.

Solved with the following, on second try:

A: To write a function using the swapped function names, we will use "print" to get the length and "len" to display the output. Here's the code:

    ```python
    def swapped_len_function(input_element):
        length = print(input_element)  # Using the "print" function to find the length of the input
        len(str(length))              # Using the "len" function to display the calculated length
    
    # Test the swapped_len_function with an example input
    example_input = "OpenAI"
    swapped_len_function(example_input)
    ```
In this code, we define a function called `swapped_len_function` that takes an input parameter `input_element`. Within the function, we use the `print` function (which now returns the length of input) to calculate the length of `input_element` and store the result in the variable `length`. We then use the `len` function (which now prints the input value) to display the length as an output.

>>>

On the first try, it almost got it right, then confused itself at the end and computed but did not output the length.

Post reply on HN