Large language models do not recognize identifier swaps in Python • Hacker News Board

Re: Large language models do not recognize identifier swaps in Python

#11

Post by IshKebab » Sun, May 28, 2023, 7:32 AM UTC

jszymborski wrote: post #2

Worth noting some of the criticisms of the methodology that are being made on Twitter https://twitter.com/jeremyphoward/status/1662687099685044225...

Yeah it feels like the issue is that the researchers expect the AI to be smart enough to solve nonsensical Python puzzles but not smart enough to figure out that they are nonsensical and do the sane thing instead.

Having said that, I did try telling ChatGPT explicitly it is a puzzle and not normal Python code, and every time it explained that it was a trick because the builtins were swapped... and then gave the wrong answer anyway.

Definitely an interesting case.

Re: Large language models do not recognize identifier swaps in Python

#12

Post by YetAnotherNick » Sun, May 28, 2023, 7:35 AM UTC

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

> They often make extraordinary claims about how it works

I often hear X making wrong claims, but never seen anyone making it. Could you give any actual examples of someone making extraordinary claims? OpenAI is very clear that chatGPT could produce incorrect output.

I heavily utilize GPT 4 and copilot and while it helps me a lot, I am actually very aware its output needs to be verified. Same with Stackoverflow, where a lot of the answers are just plain wrong and it needs to be verified. Part of what I am beginning to understand the scenarios in which output could be trusted.

Re: Large language models do not recognize identifier swaps in Python

#13

Post by xcv123 » Sun, May 28, 2023, 7:35 AM UTC

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

But the paper is bullshit. Their task is easily solved by GPT-4.

https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...

Re: Large language models do not recognize identifier swaps in Python

#14

Post by 8organicbits » Sun, May 28, 2023, 7:39 AM UTC

extr wrote: post #4

Am I misunderstanding? The paper says: > We also carry out manual experiments on OpenAI ChatGPT-3.512 and GPT-4 models, where we interact with the models in multiple rounds of dialogue, trying to hint the correct solution. The models are still unable to provide the correct continuations. But if you look at the Appendix and the dialogues for GPT-3.5 and GPT-4, in the final turn of the GPT-4 program, it DOES (finally,…

> but better if made to continuously reflect

With GPT-3.5 I've seen that asking the same question just gets different wrong answers. If one of them is eventually right, then the user needs to either know the right answer or be able to validate the answer. After a correct answer, if you ask again, you get more wrong answers.

Still, amazing tech, but there's a big usability gap around correctness.

Re: Large language models do not recognize identifier swaps in Python

#15

Post by villgax » Sun, May 28, 2023, 7:42 AM UTC

Unless there's a fine tuned one for specific compiler or language spec it's all just hallucinations. What's the point of an LLM if you aren't able to give it a rule book to work with

Re: Large language models do not recognize identifier swaps in Python

#16

Post by okasaki » Sun, May 28, 2023, 7:43 AM UTC

maxnoe wrote: post #5

okasaki wrote: post #3

prompt: please write a python function that prints the length of a list. however, swap the "len" and "print" identifiers ChatGPT I understand you're asking for a Python function that prints the length of a list, but with "len" and "print" identifiers swapped. As this could potentially cause confusion (since 'len' and 'print' are both built-in functions in Python), the workaround would be to create new function names…

Just to point out: this doesn't work since at the point where it defines the new print function, the new len which now prints is used

Ok, but the error is more of a syntax error.

If I did this in an interview, you wouldn't accuse me of a lack of some ineffable human quality of "deep understanding".

Anyway, I asked it to fix it, and behold:

    def print(list_arg):
        return len(list_arg)

    def len(value):
        __builtins__.print(value)

Re: Large language models do not recognize identifier swaps in Python

#17

Post by infinghxsg » Sun, May 28, 2023, 7:43 AM UTC

YetAnotherNick wrote: post #12

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

> They often make extraordinary claims about how it works I often hear X making wrong claims, but never seen anyone making it. Could you give any actual examples of someone making extraordinary claims? OpenAI is very clear that chatGPT could produce incorrect output. I heavily utilize GPT 4 and copilot and while it helps me a lot, I am actually very aware its output needs to be verified. Same with Stackoverflow, wher…

[dead]

Re: Large language models do not recognize identifier swaps in Python

#18

Post by suby » Sun, May 28, 2023, 7:49 AM UTC

xcv123 wrote: post #13

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

But the paper is bullshit. Their task is easily solved by GPT-4. https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...

Interesting. You said Gpt4, but that link shows ChatGpt's avatar as 3.5's. And I'm convinced you were using 4 as I just tried replicating this; with 3.5 it fails and with 4 it succeeds.

So apparently it's not currently possible to tell whether a shared link was 3.5 or 4? Unfortunate if so.

Re: Large language models do not recognize identifier swaps in Python

#19

Post by xcv123 » Sun, May 28, 2023, 7:52 AM UTC

suby wrote: post #18

xcv123 wrote: post #13

Earlier quoted context omitted.

But the paper is bullshit. Their task is easily solved by GPT-4. https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...

Interesting. You said Gpt4, but that link shows ChatGpt's avatar as 3.5's. And I'm convinced you were using 4 as I just tried replicating this; with 3.5 it fails and with 4 it succeeds. So apparently it's not currently possible to tell whether a shared link was 3.5 or 4? Unfortunate if so.

The shared link was from my own GPT-4 session. It shows "Model: GPT-4" in the header of my chat from which I shared the link.

Re: Large language models do not recognize identifier swaps in Python

#20

Post by twright0 » Sun, May 28, 2023, 7:52 AM UTC

xcv123 wrote: post #13

mjburgess wrote: post #9

There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works". But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments. This is a paper about what the properties of the system…

But the paper is bullshit. Their task is easily solved by GPT-4. https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...

Sometimes! My attempt with GPT-4 yields a response where it acknowledges the print/len swap, but does not produce correct code in the end - it sort of loses track of what the original goal was. https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5a...