Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs understand nullability

dmodel.ai

Post reply on HN
Print view View original

51–60 of 143 posts

Re: LLMs understand nullability

#51

Post by delusional »

gr_norm wrote: post #33
wongarsu wrote: post #30

Earlier quoted context omitted.

We spent 40 years moving the goal posts on what constitutes AI. Now we seem to have found an AI worthy of that title and instead start moving the goal posts on "consciousness", "understanding" and "intelligence".

Indeed, science is a process of discovery and adjusting goals and expectations. It is not a mountain to be summited. It is highly telling that the LLM boosters do not understand this. Those with a genuine interest in pushing forward our understanding of cognition do.

They believe that once they reach this summit everything else will be trivial problems that can be posed to the almighty AI. It's not that they don't understand the process, it's that they think AI is going to disrupt that process.

They literally believe that the AI will supersede the scientific process. It's crypto shit all over again.

Re: LLMs understand nullability

#52

Post by gopiandcode »

qsort wrote: post #9
gopiandcode wrote: post #5

The visualisation of how the model sees nullability was fascinating. I'm curious if this probing of nullability could be composed with other LLM/ML-based python-typing tools to improve their accuracy. Maybe even focusing on interfaces such as nullability rather than precise types would work better with a duck-typed language like python than inferring types directly (i.e we don't really care if a variable is an int sp…

> we don't really care if a variable is an int specifically, but rather that it supports _add or _sub etc. that it is numeric my brother in christ, you invented Typescript. (I agree on the visualization, it's very cool!)

I am more than aware of Typescript, you seem to have misunderstood my point: I was not describing a particular type system (of which there have been many of this ilk) but rather conjecturing that targeting interfaces specifically might make LLM-based code generation/type inference more effective.

Re: LLMs understand nullability

#53

Post by Sohcahtoa82 »

mubou wrote: post #25
gwd wrote: post #20

Earlier quoted context omitted.

> Each word/token is chosen as that which is statistically most likely to follow the previous. The best way to predict the weather is to have a model which approximates the weather. The best way to predict the results of a physics simulation is to have a model which approximates the physical bodies in question. The best way to predict what word a human is going to write next is to have a model that approximates human…

LLMs don't approximate human thought , though. They approximate language . That's it. Please, I'm begging you, go read some papers and watch some videos about machine learning and how LLMs actually work. It is not "thinking." I fully realize neural networks can approximate human thought -- but we are not there yet, and when we do get there, it will be something that is not an LLM, because an LLM is not capable of tha…

> it will be something that is not an LLM

I think it will be very similar in architecture.

Artificial neural networks already are approximating how neurons in a brain work, it's just at a scale that's several orders of magnitude smaller.

Our limiting factor for reaching brain-like intelligence via ANN is probably more of a hardware limitation. We would need over 100 TB to store the weights for the neurons, not to mention the ridiculous amount of compute to run it.

Re: LLMs understand nullability

#54

Post by redundantly »

delusional wrote: post #51
gr_norm wrote: post #33

Earlier quoted context omitted.

Indeed, science is a process of discovery and adjusting goals and expectations. It is not a mountain to be summited. It is highly telling that the LLM boosters do not understand this. Those with a genuine interest in pushing forward our understanding of cognition do.

They believe that once they reach this summit everything else will be trivial problems that can be posed to the almighty AI. It's not that they don't understand the process, it's that they think AI is going to disrupt that process. They literally believe that the AI will supersede the scientific process. It's crypto shit all over again.

Well, if that summit were reached and AI is able to improve itself trivially, I'd be willing to cede that they've reached their goal.

Anything less than that, meh.

Re: LLMs understand nullability

#55

Post by nick__m »

plaineyjaney wrote: post #29

This is really interesting! Intuitively it's hard to grasp that you can just subtract two average states and get a direction describing the model's perception of nullability.

The original word2vec example might be easier to understand:

  vec(King) - vec(Man) + vec(Woman) = vec(Queen)

Re: LLMs understand nullability

#56

Post by joe8756438 »

darkerside wrote: post #41
bluefirebrand wrote: post #36

Earlier quoted context omitted.

> Now we seem to have found an AI worthy of that title and instead start moving the goal posts on "consciousness", "understanding" and "intelligence". We didn't "find" AI, we invented systems that some people want to call AI, and some people aren't convinced it meets the bar It is entirely reasonable for people to realize we set the bar too low when it is a bar we invented

What should the bar be? Should it be higher than it is for the average human? Or even the least intelligent human?

there is no such bar.

We don’t even have a good way to quantify human ability. The idea that we could suddenly develop a technique to quantify human ability because we now have a piece of technology that would benefit from that quantification is absurd.

That doesn’t mean we shouldn’t try to measure the ability of an LLM. But it does mean that the techniques used to quantify an LLMs ability are not something that can be applied to humans outside of narrow focus areas.

Re: LLMs understand nullability

#57

Post by 6510 »

wongarsu wrote: post #30
nativeit wrote: post #28

Earlier quoted context omitted.

Sorry, this is more about the discussion of this article than the article itself. The moving goal posts that acolytes use to declare consciousness are becoming increasingly cult-y.

We spent 40 years moving the goal posts on what constitutes AI. Now we seem to have found an AI worthy of that title and instead start moving the goal posts on "consciousness", "understanding" and "intelligence".

My joke was that the what it cant do debate changed into what it shouldn't be allowed to.

Re: LLMs understand nullability

#58

Post by stefl14 »

nativeit wrote: post #21
nsingh2 wrote: post #6

Earlier quoted context omitted.

Where is the boundary where this becomes semantically correct? It's easy for these kinds of discussions to go in circles, because nothing is well defined.

Hard to define something that science has yet to formally outline, and is largely still in the realm of religion.

Shameless plug of personal blog post, but relevant. Still not fully edited, so writing is a bit scattered, but crux is we now have the framework for talking about consciousness intelligently. It's not as mysterious as in the past, considering advances in non-equilibrium thermodynamics and the Free Energy Principle in particular.

https://stefanlavelle.substack.com/p/i-am-therefore-i-feel

Re: LLMs understand nullability

#59

Post by bluefirebrand »

darkerside wrote: post #41
bluefirebrand wrote: post #36

Earlier quoted context omitted.

> Now we seem to have found an AI worthy of that title and instead start moving the goal posts on "consciousness", "understanding" and "intelligence". We didn't "find" AI, we invented systems that some people want to call AI, and some people aren't convinced it meets the bar It is entirely reasonable for people to realize we set the bar too low when it is a bar we invented

What should the bar be? Should it be higher than it is for the average human? Or even the least intelligent human?

Personally I don't care what the bar is, honestly

Call it AI, call it LLMs, whatever

Just as long as we continue to recognize that it is a tool that humans can use, and don't start trying to treat it as a human, or as a life, and I won't complain

I'm saving my anger for when idiots start to argue that LLMs are alive and deserve human rights

Re: LLMs understand nullability

#60

Post by stared »

Once LLMs fully understand nullability, they will cease to use that.

Tony Hoare called it "a billion-dollar mistake" (https://en.wikipedia.org/wiki/Tony_Hoare#Apologies_and_retra...), Rust had made core design choices precisely to avoid this mistake.

In practical AI-assisted coding in TypeScript I have found that it is good to add in Cursor Rules to avoid anything nullable, unless it is a well-designed choice. In my experience, it makes code much better.

Post reply on HN