I found this overly handwavy, but I discovered that there is a non-"gentle" version of this page which is more explicit: https://dmodel.ai/nullability/
Yeah that's linked a couple of times in the post
LLMs understand nullability
111–120 of 143 posts
Re: LLMs understand nullability
#112I'm curious what happens if you run the LLM with variable names that occur often with nullable variables, but then use them with code that has a non-nullable variable.
The answer it seems is, it depends on what kind of code you're looking at. The post showed that `for` loops cause a lot more variable-name-biased reasoning, while `ifs` and function defs/calls are more variable-name independent.
Re: LLMs understand nullability
#113The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…
This is essentially John Searle's Chinese Room Argument against strong AI. His conclusion is broader -- he argues categorically against the very possibility of so-called "strong" AI, viz. AI that understands, not just against the narrower notion that LLMs "understand" -- but the reasoning is essentially identical.
Here's the Stanford Encyclopedia of Philosophy's superb write up: https://plato.stanford.edu/entries/chinese-room/ It covers, engagingly and with playful wit, not just Searle's original argument but its evolution in his writing, other philosopher's responses/criticisms, and Searle's counter-responses.
Re: LLMs understand nullability
#114Earlier quoted context omitted.
LLMs don't approximate human thought , though. They approximate language . That's it. Please, I'm begging you, go read some papers and watch some videos about machine learning and how LLMs actually work. It is not "thinking." I fully realize neural networks can approximate human thought -- but we are not there yet, and when we do get there, it will be something that is not an LLM, because an LLM is not capable of tha…
> it will be something that is not an LLM I think it will be very similar in architecture. Artificial neural networks already are approximating how neurons in a brain work, it's just at a scale that's several orders of magnitude smaller. Our limiting factor for reaching brain-like intelligence via ANN is probably more of a hardware limitation. We would need over 100 TB to store the weights for the neurons, not to men…
How much compute do you think the human brain uses? They're training these LLMs with (hundreds of) thousands of GPUs.
Re: LLMs understand nullability
#115Earlier quoted context omitted.
I was going to say "so you believe the LLM's don't have the capacity to understand" but then I realized that the precise language would be something like "the presence of photons in this human's retinas in patterns encoding statements about LLM's having understanding correlates to the activation of neuron signaling chains corresponding to, and emission of, muscle activations engaging keyboard switches, which produce…
No it's not. He gave you modal conditions on "understanding", he said: predicting the syntax of valid programs, and their operational semantics, ie., the behaviour of the computer as it runs. I would go much further than this; but this is a de minimus criteria that the LLM already fails. What zealots eventually discover is that they can hold their "fanatical proposition" fixed in the face of all opposition to the con…
>No it's not. He gave you modal conditions on "understanding", he said: predicting the syntax of valid programs, and their operational semantics, ie., the behaviour of the computer as it runs.
LLMs are perfectly capable of predicting the behavior of programs. You don't have to take my word for it, you can test it yourself. So he gave modal conditions they already satisfy. Can I conclude they understand now ?
>If you find yourself hollowing-out the meaning of words to the point of making no distinctions, denying reality to reality itself, and otherwise arriving at a "philosophical abyss" be aware that it is your cherished propositions which are the maddness and nothing else.
The only people denying reality are those who insist that it is not 'real' understanding and yet cannot distinguish this 'real' from 'fake' property in any verifiable manner, the very definition of an invented property.
Your argument boils down to 'I think it's so absurd so it cannot be so'. That's the best you can do ? Do you genuinely think that's a remotely convincing argument ?
Re: LLMs understand nullability
#116Earlier quoted context omitted.
I was going to say "so you believe the LLM's don't have the capacity to understand" but then I realized that the precise language would be something like "the presence of photons in this human's retinas in patterns encoding statements about LLM's having understanding correlates to the activation of neuron signaling chains corresponding to, and emission of, muscle activations engaging keyboard switches, which produce…
Countering the argument that LLMs are just gloriefied probability machines and do not undertand or think with "how do you know humans are not the same" has been the biggest achievement of AI hypemen (and yes, it's mostly men). Of course, now you can say "how do you know that our brains are not just efficient computers that run LLMs", but I feel like the onus of proof lies on the makers of this claim, not on the other…
I’ve come to realize AI works as well as it does because it was trained extensively on the same kinds of things people normally ask. So, it already has the benefit of vast amounts of human responses.
Of course, ask it a PhD level question and it will confidently hallucinate more than Beavis & Butthead.
It really is a damn glorified autocomplete, unfortunately very useful as a search engine replacement.
Re: LLMs understand nullability
#117Earlier quoted context omitted.
No it's not. He gave you modal conditions on "understanding", he said: predicting the syntax of valid programs, and their operational semantics, ie., the behaviour of the computer as it runs. I would go much further than this; but this is a de minimus criteria that the LLM already fails. What zealots eventually discover is that they can hold their "fanatical proposition" fixed in the face of all opposition to the con…
>No it's not. He gave you modal conditions on "understanding", he said: predicting the syntax of valid programs, and their operational semantics, ie., the behaviour of the computer as it runs. LLMs are perfectly capable of predicting the behavior of programs. You don't have to take my word for it, you can test it yourself. So he gave modal conditions they already satisfy. Can I conclude they understand now ? >If you…
LLMs are reasonably competent at surfacing the behaviour of simple programs when the behaviour of those programs is a relatively straightforward structural extension of enough of its training set that it's managed to correlate together.
It's very clear that LLMs lack understanding when you use them for anything remotely sophisticated. I say this as someone who leverages them extensively on a daily basis - mostly for development. They're very powerful tools and I'm grateful for their existence and the force multiplier they represent.
Try to get one to act as a storyteller and the limitations in understanding glare out. You try to goad some creativity and character out of it and it spits out generally insipid recombinations of obvious tropes.
In programming, I use AI strictly as a auto-complete extension. Even in that limited context, the latest models make trivial mistakes in certain circumstances that reveal their lack of understanding. The ones that stand out are the circumstances where the local change to make is very obvious and simple, but the context of the code is something that the ML hasn't seen before.
In those cases, I see them slapping together code that's semantically wrong in the local context, but pattern matches well against the outer context.
It's very clear that the ML doesn't even have a SIMPLE understanding of the language semantics, despite having been trained on presumably multiple billions of lines of code from all sorts of different programming languages.
If you train a human against half a dozen programming languages, you can readily expect by the end of that training that they will, all by themselves, simply through mastering the individual languages, have constructed their own internal generalized models of programming languages as a whole, and would become aware of some semantic generalities. And if I had asked that human to make that same small completion for me, they would have gotten it right. They would have understood that the language semantics are a stronger implicit context compared to the surrounding syntax.
MLs just don't do that. They're very impressive tools, and they are a strong step forward toward some machine model of understanding (sophisticated pattern matching is likely a fundamental prerequisite for understanding), but ascribing understanding to them at this point is jumping the gun. They're not there yet.
Re: LLMs understand nullability
#118Earlier quoted context omitted.
No, most people absolutely use non-linguistic, involuntary cues when judging the responses of other people. To not do that is commonly associated with things like being on the spectrum or cognitive deficiencies.
The fact we have labels for communication problems caused by failure to understand non-verbal cues doesn’t tell us that non-verbal cues are necessary for understanding
I didn't say they are always necessary so you probably replied in the wrong place.
Re: LLMs understand nullability
#119Earlier quoted context omitted.
No, most people absolutely use non-linguistic, involuntary cues when judging the responses of other people. To not do that is commonly associated with things like being on the spectrum or cognitive deficiencies.
On a message board? Do you have theories about whether people on this thread understand or don't understand what they're talking about?
Why would you add this constraint now?
Re: LLMs understand nullability
#120Earlier quoted context omitted.
No, most people absolutely use non-linguistic, involuntary cues when judging the responses of other people. To not do that is commonly associated with things like being on the spectrum or cognitive deficiencies.
You are saying no while presenting nothing to contradict what GP said. Judging someone's external "involuntary cues" is not interrogating their internal state. It is, as you said, judging their response (a synonym for "answer") - and that judgment is also highly imperfect. (It's worth noting that focusing so much on the someone's body language and tone that you ignore the actual words they said is a communication iss…
Are you sure?
"Nobody interrogates each other's internal states when judging whether someone understands a topic. All we can judge it based on are the words they produce or the actions they take in response to a situation."
I'm fairly sure I wrote something that contradicts these two sentences.
You can commonly figure out whether people are leaking information about how they feel and what emotional and cognitive states they are in, even in text based communication. To some extent you can also decide the internal state of another person, and it's often easier to do with text than in direct communication, i.e. two bodies in the same space.
This idea that people aren't malleable and possible to interrogate without them noticing would be very surprising to anyone making a living from advertising.