Large language models do not recognize identifier swaps in Python • Hacker News Board

Re: Large language models do not recognize identifier swaps in Python

#101

Post by starbugs » Mon, May 29, 2023, 3:12 PM UTC

TeMPOraL wrote: post #97

starbugs wrote: post #91

Earlier quoted context omitted.

I am frequently talking to people outside the tech bubble. Could you give me an example? And maybe a hint how this would relate to my argument about intention and understanding? I have a hard time following your argument.

Tell your family and friends you, e.g., like regularly having a pizza for dinner, or that you're ditching sugar for artificial sweetener - and enjoy watching people on an acid trip with no LSD. You'll hear plenty of warnings warning and advice that's quite obviously bullshit, and which they acquired from other people and/or lifestyle magazines/blogs/memes - except they don't even realize it, because they're not in a…

Again, my point is not about people (maybe wrongly) reciting sentences that they have picked up somewhere without any attempt at verification or understanding of the matter. My point is making up entire new entities (function names) that don't exist at all and claiming these things as facts in a very self-confident tone. While mindlessly reciting "facts" that you have picked up somewhere as a conversational style doesn't require any understanding (it's called belief), making up entities demonstrates that you do something that requires a real understanding, but you don't have that understanding. You just somewhat carefully arrange words based on statistics to match a query. I've yet to find a human who does this. Even my uncle who fits your description more than perfectly wouldn't do this. And I've never met anyone who is mentally healthy and sober doing this.

Re: Large language models do not recognize identifier swaps in Python

#102

Post by starbugs » Mon, May 29, 2023, 3:25 PM UTC

TeMPOraL wrote: post #94

starbugs wrote: post #87

Earlier quoted context omitted.

> str.cou > str.size() and > str.split > does not have a split() method. Both confuse two existing entities from different languages/frameworks, which is exactly my point. These entities exist, they are not made up at all. They are just out of context. Ironically, the LLMs I used did rarely run into this kind of mistake (confusing similar entities of a different context). As an example of what I got when asking a pop…

> It lectures me in a quite self-satisfied tone that I should commence to use the function "AXObserve" that does not exist, as it turns out. This is completely made up. There is no such thing. If you google the exact word, it doesn't even bring up anything related to programming let alone the macOS APIs. I've seen this happening too, but it still feel more like a case of a generalized "there is something like this, o…

> Sure. I suppose we should also agree on the same understanding of "real understanding". I'm only proposing that LLMs pick up the same kind of "understanding" your unconscious/subconscious mind does, and produce output of similar nature. This implies that, to replicate human reasoning/performance, we'll need to layer some additional models/systems on top of the LLM.

As you might probably guess, I am not convinced. It may be part of something that can match up to human intelligence. But is it enough to layer more mechanisms on top of it? I am not sure.

I think the real question is how to define "real understanding" as you pointed out. I am not sure this will be possible using language alone. Also I think it will probably be hard to compare it to the human mind in a scientific sense, since we don't know how that works and we have no way of knowing other than collecting anecdotes like those that we came up with in our comments here.

Re: Large language models do not recognize identifier swaps in Python

#103

Post by rad88 » Mon, May 29, 2023, 9:24 PM UTC

mjburgess wrote: post #42

8organicbits wrote: post #14

Earlier quoted context omitted.

> but better if made to continuously reflect With GPT-3.5 I've seen that asking the same question just gets different wrong answers. If one of them is eventually right, then the user needs to either know the right answer or be able to validate the answer. After a correct answer, if you ask again, you get more wrong answers. Still, amazing tech, but there's a big usability gap around correctness.

> there's a big usability gap around correctness From an engineer's perspective I can see this seems minor -- but from a scientific perspective, it's kinda a crazy statement, right? Imagine an alien who speaks english giving apparently correct descriptions of, eg., a room; and then, seemingly at random, saying wholly false things with the confidence (etc.) of its other statements. A scientist studying this alien woul…

It doesn't seem that crazy. Working from a simpler analogy first -- if an alien can read 95% of written English, but completely mischaracterizes the remaining 5%, would a reasonable scientist conclude that they can't read at all and are just guessing randomly? No, and it would indeed be unreasonable to conclude that there's no way to write differently so that the alien performs substantially better.

We know some more about LLMs than we do aliens. The LLM is a neural network whose parameters have been optimized to reduce the total size of its errors, as measured against an enormous set of empirical data.

We have to add to your analogy then, that it's somehow known the alien speaks English on their own planet remarkably well. They are still not perfectly correct, but when they have to describe something on that planet, in English, they can do it better than they generally can given the same task on Earth.

It would again be totally unscientific to conclude that because of this, there is no way the alien can talk about earthly objects or ideas. The impulse of the mere engineer, to just hack around and find out, is a scientific one.