Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs understand nullability

dmodel.ai

Post reply on HN
Print view View original

81–90 of 143 posts

Re: LLMs understand nullability

#81

Post by uh_uh »

lsy wrote: post #74

The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…

We don't really have a clue what they are and aren't capable of. Prior to the LLM-boom, many people – and I include myself in this – thought it'd be impossible to get to the level of capability we have now purely from statistical methods and here we are. If you have a strong theory that proves some bounds on LLM-capability, then please put it forward. In the absence of that, your sceptical attitude is just as sus as the article's.

Re: LLMs understand nullability

#82

Post by LordDragonfang »

cess11 wrote: post #73
empath75 wrote: post #70

Earlier quoted context omitted.

Is there any way you can tell whether a human understands something other than by asking them a question and judging their answer? Nobody interrogates each other's internal states when judging whether someone understands a topic. All we can judge it based on are the words they produce or the actions they take in response to a situation. The way that systems or people arrive at a response is sort of an implementation…

No, most people absolutely use non-linguistic, involuntary cues when judging the responses of other people. To not do that is commonly associated with things like being on the spectrum or cognitive deficiencies.

You are saying no while presenting nothing to contradict what GP said.

Judging someone's external "involuntary cues" is not interrogating their internal state. It is, as you said, judging their response (a synonym for "answer") - and that judgment is also highly imperfect.

(It's worth noting that focusing so much on the someone's body language and tone that you ignore the actual words they said is a communication issue associated with not being on the spectrum, or being too allistic)

Re: LLMs understand nullability

#83

Post by wvenable »

lsy wrote: post #74

The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…

> Because code LLMs have been trained on the syntactic form of the program and not its execution

One of the very first tests I did of ChatGPT way back when it was new was give it a relatively complex string manipulation function from our code base, strip all identifying materials from the code (variable names, the function name itself, etc), and then provide it with inputs and ask it for the outputs. I was surprised that it could correctly generate the output from the input.

So it does have some idea of what the code actually does not just syntax.

Re: LLMs understand nullability

#84

Post by SJC_Hacker »

jquery wrote: post #71
SJC_Hacker wrote: post #19

Earlier quoted context omitted.

Thats probably the case 99% of the time. But that 1% is pretty important. For example, they are dismal at math problems that aren't just slight variations of problems they've seen before. Here's one by blackandredpenn where ChatGPT insisted the solution to problem that could be solved by high school / talented middle school students was correct, even after trying to convince it it was wrong. https://youtu.be/V0jhP7gi…

ChatGPT o1 pro mode solved it on the first try, after 8 minutes and 53 seconds of “thinking”: https://chatgpt.com/share/67f40cd2-d088-8008-acd5-fe9a9784f3...

The problem is how do you know that its correct ...

A human would probably say "I don't know how to solve the problem". But ChatGPT free version is confidentially wrong ..

Re: LLMs understand nullability

#85

Post by waldrews »

lsy wrote: post #74

The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…

I was going to say "so you believe the LLM's don't have the capacity to understand" but then I realized that the precise language would be something like "the presence of photons in this human's retinas in patterns encoding statements about LLM's having understanding correlates to the activation of neuron signaling chains corresponding to, and emission of, muscle activations engaging keyboard switches, which produce patterns of 'no they don't' with high frequency."

The critiques of mental state applied to the LLM's are increasingly applicable to us biologicals, and that's the philosophical abyss we're staring down.

Re: LLMs understand nullability

#87

Post by yujzgzc »

lsy wrote: post #74

The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…

How do you know that these models haven't been trained by running programs?

At least, it's likely that they've been trained on undergrad textbooks that explain program behaviors and contain exercises.

Re: LLMs understand nullability

#88

Post by aSanchezStern »

tanvach wrote: post #66

Dear future authors: please run multiple iterations and report the probability . From: ‘Keep training it, though, and eventually it will learn to insert the None test’ To: ‘Keep training it, though, and eventually the probability of inserting the None test goes up to xx%’ The former is just horse poop, we all know LLMs generate big variance in output.

If you're interested in a more scientific treatment of the topic, the post links to a technical report which reports the numbers in detail. This post is instead an attempt to explain the topics to a more general audience, so digging into the weeds isn't very useful.

Re: LLMs understand nullability

#89

Post by [deleted] »

uh_uh wrote: post #81
lsy wrote: post #74

The article puts scare quotes around "understand" etc. to try to head off critiques around the lack of precision or scientific language, but I think this is a really good example of where casual use of these terms can get pretty misleading. Because code LLMs have been trained on the syntactic form of the program and not its execution, it's not correct — even if the correlation between variable annotations and request…

We don't really have a clue what they are and aren't capable of. Prior to the LLM-boom, many people – and I include myself in this – thought it'd be impossible to get to the level of capability we have now purely from statistical methods and here we are. If you have a strong theory that proves some bounds on LLM-capability, then please put it forward. In the absence of that, your sceptical attitude is just as sus as…

[deleted]

Re: LLMs understand nullability

#90

Post by gwd »

mubou wrote: post #25
gwd wrote: post #20

Earlier quoted context omitted.

> Each word/token is chosen as that which is statistically most likely to follow the previous. The best way to predict the weather is to have a model which approximates the weather. The best way to predict the results of a physics simulation is to have a model which approximates the physical bodies in question. The best way to predict what word a human is going to write next is to have a model that approximates human…

LLMs don't approximate human thought , though. They approximate language . That's it. Please, I'm begging you, go read some papers and watch some videos about machine learning and how LLMs actually work. It is not "thinking." I fully realize neural networks can approximate human thought -- but we are not there yet, and when we do get there, it will be something that is not an LLM, because an LLM is not capable of tha…

> LLMs don't approximate human thought, though. ...Please, I'm begging you, go read some papers and watch some videos about machine learning and how LLMs actually work.

I know how LLMs work; so let me beg you in return, listen to me for a second.

You have a theoretical-only argument: LLMs do text prediction, and therefore it is not possible for them to actually think. And since it's not possible for them to actually think, you don't need to consider any other evidence.

I'm telling you, there's a flaw in your argument: In actuality, the best way to do text prediction is to think. An LLM that could actually think would be able to do text prediction better than an LLM that can't actually think; and the better an LLM is able to approximate human thought, the better its predictions will be. The fact that they're predicting text in no way proves that there's no thinking going on.

Now, that doesn't prove that LLMs actually are thinking; but it does mean that they might be thinking. And so you should think about how you would know if they're actually thinking or not.

Post reply on HN