LLMs understand nullability • Hacker News Board

Re: LLMs understand nullability

#141

Post by uh_uh » Tue, Apr 08, 2025, 6:24 PM UTC

kubav027 wrote: post #135

uh_uh wrote: post #122

Earlier quoted context omitted.

1. Many humans don't have an idea of the limits of their competence. It's called the Dunning–Kruger effect. 2. LLMs regularly tell me if what I'm asking for is possible or not. I'm not saying they're always correct, but they seem to have at least some sense of what's in the realm of possibility.

1. Dunning-kruger effect describes difference in expected and real performance. It is not saying that humans confidently give wrong answers if they do not know correct ones. 2. That is not my experience. Almost half of the time LLM gives wrong answer without any warning. It is up to me to check correctness. Even if I follow up it often continues to give wrong answers.

1. "It is not saying that humans confidently give wrong answers if they do not know correct ones." And I didn't say that they do either, so you might have hallucinated that.

2. What are you arguing about? I didn't say they're always correct obviously.

Re: LLMs understand nullability

#142

Post by shafyy » Wed, Apr 09, 2025, 12:02 PM UTC

uh_uh wrote: post #140

shafyy wrote: post #132

Earlier quoted context omitted.

> Do you have a theory about when certain LLM capabilities emerge? We do know how LLMs work, correct? We also know what they are capable of and what not (of course this line is often blurred by hype). I am not an expert at all on LLMs or neuroscience. But it is apparent that having a discussion with a human vs. with an LLM is a completely different ballpark. I am not saying that we will never have technology that can…

> We do know how LLMs work, correct? NO! We have working training algorithms. We still don't have a complete understanding of why deep learning works in practice, and especially not why it works at the current level of scale. If you disagree, please cite me the papers because I'd love to read them. To put in another way: Just because you can breed dogs, it doesn't necessary mean that you have a working theory of gene…

We do know in general how LLMs work. Now, it's of course not always possible to say why a specific output is generated given an input, but we do know HOW it does it.

To translate it to your analogy with dogs: We do know how the anatomy of dogs work, but we do not know why they sometimes fetch the stick and sometimes not.

Re: LLMs understand nullability

#143

Post by 6510 » Thu, Apr 10, 2025, 6:35 PM UTC

wizardforhire wrote: post #64

6510 wrote: post #57

Earlier quoted context omitted.

My joke was that the what it cant do debate changed into what it shouldn't be allowed to.

There ARE no jokes aloud on hn. Look I’m no stranger to love. you know the rules and so do I… you can’t find this conversation with any other guy. But since the parent was making a meta commentary on this conversation I’d like to introduce everyone here as Kettle to a friend of mine known as #000000

Is there reason to think language developers understand nullability?