Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do LLMs have emergent properties?

johndcook.com

Post reply on HN
Print view View original

111–120 of 121 posts

Re: Why do LLMs have emergent properties?

#111

Post by PaulDavisThe1st »

pixl97 wrote: post #87
PaulDavisThe1st wrote: post #67

Earlier quoted context omitted.

To return an old but still good analogy ... If you want to understand how birds fly, the fact that planes also fly is near useless. While a few common aerodynamic principles apply, both types of flight are so different from each other that you do not learn very much about one from the other. On the other hand, if your goal is just "humans moving through the air for extended distances", it doesn't matter at all that a…

doesn't by itself tell you that they are working in remotely the same way or capable of the same sorts of things. I believe any intelligence that reaches 'human level' should be capable of nearly the same things with tool use, the fact it accomplishes the goal in a different way doesn't matter because the systems behavior is generalized. Hence the term (artificial) general intelligence. Two different general intellig…

There is no "human level" because we don't even understand what we mean by "human level". We don't know what metrics to use, we don't even know what to measure.

> Two different general intelligences built on different architectures should be able to converge on similar solutions (for example solutions based on lowest energy states) because they are operating in the same physical realm.

Lots of things connected to intelligence do not operate (much) in any physical realm.

Also, you've really missed the point of the analogy. It's not a question of whether AGI would pick the same solution for fast air travel as HGI. It is that there are least two solutions to the challenge of moving things through the air in a controlled way, and they don't really work in the same way at all. Consequently, we should be ready for the possibility that there is more than one way to do the things LLMs (and to some degree) humans do with text/language, and that they may not be related to each very much. This is a counter to the claim that "since LLMs get so close to human language behavior, it seems quite likely human language behavior arises from a system like an LLM".

Re: Why do LLMs have emergent properties?

#112

Post by bunderbunder »

drdeca wrote: post #98
andy99 wrote: post #78

Earlier quoted context omitted.

An LLM is a classifier, there is lots of research into how deep learning classifiers work, that I haven't seen contradicted when applied to LLMs.

I still think it seems unclear what you mean by “interpolate” in this context? If your NN takes in several numbers and assigns logits to each class based on those numbers, then if you consider the n dimensional space of possible inputs, and if the new input is in the convex hull of the inputs that appear in training samples, then the meaning of “interpolate” is fairly clear. But when the inputs are sequences of token…

The answer is kind of right there in the start of your last sentence. From the transformer model's perspective, the input is just a time series of vectors. It ultimately isn't any different from any other time series of vectors.

Way back in the day when I was working with latent Dirichlet allocation models, I had a minor enlightenment moment when I realized that the models really weren't capturing any semantically meaningful relationships. They were only capturing meaningless statistical correlations to which I would then assign semantic value so effortlessly and automatically that I didn't even realize it was always me doing it, never the model.

I'm pretty sure LLMs exist on that same continuum. And if you travel down it in the other direction, you get to simple truisms such as "correlation does not equal causation."

Re: Why do LLMs have emergent properties?

#113

Post by vonneumannstan »

Der_Einzige wrote: post #105
vonneumannstan wrote: post #20

Earlier quoted context omitted.

This doesn't seem right and most people recognize that 'neurons' encode for multiple activations. https://transformer-circuits.pub/2022/toy_model/index.html

They’re 1000% right on the idea that most models are hilariously undertrained

Pretty sure that since the Chinchilla paper this probably isn't the case. https://arxiv.org/pdf/2203.15556

Re: Why do LLMs have emergent properties?

#114

Post by drdeca »

bunderbunder wrote: post #112
drdeca wrote: post #98

Earlier quoted context omitted.

I still think it seems unclear what you mean by “interpolate” in this context? If your NN takes in several numbers and assigns logits to each class based on those numbers, then if you consider the n dimensional space of possible inputs, and if the new input is in the convex hull of the inputs that appear in training samples, then the meaning of “interpolate” is fairly clear. But when the inputs are sequences of token…

The answer is kind of right there in the start of your last sentence. From the transformer model's perspective, the input is just a time series of vectors. It ultimately isn't any different from any other time series of vectors. Way back in the day when I was working with latent Dirichlet allocation models, I had a minor enlightenment moment when I realized that the models really weren't capturing any semantically me…

The part about “is it in the convex hull?” was an important part of the question.

It seems to me that if it isn’t in the convex hull, it could be more fitting to describe it as extrapolation, rather than interpolation?

In general, my question does apply to the task of predicting how a time series of vectors continues: Given a dataset of time series, where the dimension of each vector in the series is such and such, the length of each series is yea long, and there are N series in the training set, should we expect series in the test set or validation set to be in the convex hull of the ones in the training set?

I would think that the number of series in the training set, N, while large, might not be all that large compared to the dimensionality of a whole series?

Hm, are there efficient techniques for evaluating whether a high dimensional vector is in the convex hull of a large number of other high dimensional vectors?

Re: Why do LLMs have emergent properties?

#115

Post by bunderbunder »

drdeca wrote: post #114
bunderbunder wrote: post #112

Earlier quoted context omitted.

The answer is kind of right there in the start of your last sentence. From the transformer model's perspective, the input is just a time series of vectors. It ultimately isn't any different from any other time series of vectors. Way back in the day when I was working with latent Dirichlet allocation models, I had a minor enlightenment moment when I realized that the models really weren't capturing any semantically me…

The part about “is it in the convex hull?” was an important part of the question. It seems to me that if it isn’t in the convex hull, it could be more fitting to describe it as extrapolation, rather than interpolation? In general, my question does apply to the task of predicting how a time series of vectors continues: Given a dataset of time series, where the dimension of each vector in the series is such and such, t…

Just shooting from the hip, LLMs operate out on a frontier where the curse of dimensionality removes a large chunk of the practical value from the concept of a convex hull. Especially in a case like this where the vector embedding process places hard limits on the range of possible magnitudes and directions for any single vector.

Re: Why do LLMs have emergent properties?

#116

Post by drdeca »

bunderbunder wrote: post #115
drdeca wrote: post #114

Earlier quoted context omitted.

The part about “is it in the convex hull?” was an important part of the question. It seems to me that if it isn’t in the convex hull, it could be more fitting to describe it as extrapolation, rather than interpolation? In general, my question does apply to the task of predicting how a time series of vectors continues: Given a dataset of time series, where the dimension of each vector in the series is such and such, t…

Just shooting from the hip, LLMs operate out on a frontier where the curse of dimensionality removes a large chunk of the practical value from the concept of a convex hull. Especially in a case like this where the vector embedding process places hard limits on the range of possible magnitudes and directions for any single vector.

Outside of the context of a convex hull, I don’t know how to make a distinction between interpolation and extrapolation. This is the core of my question.

What precisely is it that you mean when you say that it is interpolating rather than extrapolating? In the only definition that I know, the one based on convex hulls, I believe it would be extrapolating rather than interpolating. But people often say it is interpolating rather than extrapolating, and I don’t know what they mean.

Re: Why do LLMs have emergent properties?

#117

Post by trash_cat »

I think the better question is to answer why do emergent properties exist in the first place.

I disagree with the premise that emergence is binary. It's not. What we determine "emergent behaviour" is partly a social concept. We decide when an LLM is good enough for us and when it "solved" something through emergent properties.

Re: Why do LLMs have emergent properties?

#118

Post by bunderbunder »

drdeca wrote: post #116
bunderbunder wrote: post #115

Earlier quoted context omitted.

Just shooting from the hip, LLMs operate out on a frontier where the curse of dimensionality removes a large chunk of the practical value from the concept of a convex hull. Especially in a case like this where the vector embedding process places hard limits on the range of possible magnitudes and directions for any single vector.

Outside of the context of a convex hull, I don’t know how to make a distinction between interpolation and extrapolation. This is the core of my question. What precisely is it that you mean when you say that it is interpolating rather than extrapolating? In the only definition that I know, the one based on convex hulls, I believe it would be extrapolating rather than interpolating. But people often say it is interpola…

I doubt they're really thinking about it in a mathematical sense when they say that. I'm guessing, for example, that "extrapolate" is meant in the more colloquial sense, which is maybe closer to "deduce" in practice.

Re: Why do LLMs have emergent properties?

#119

Post by xboxnolifes »

tinix wrote: post #64
xboxnolifes wrote: post #63

Earlier quoted context omitted.

Has emergent behavior ever been predicted prior to it being observed in other theoretical fields?

emergent behavior is common in all large systems. it doesn't seem that surprising to me.

That is not the question.

Re: Why do LLMs have emergent properties?

#120

Post by albertzeyer »

john-h-k wrote: post #77
andy99 wrote: post #69

Earlier quoted context omitted.

Yes, deep learning models only interpolate, and essentially represent an effective way of storing data labeling effort. Doesn't mean they're not useful, just not what tech adjacent promoters want people to think.

> Yes, deep learning models only interpolate What do you mean by this? I don’t think the understanding of LLMs is sufficient to make this claim

Just by the mathematical definition of interpolation (https://en.wikipedia.org/wiki/Interpolation), any function which approximates the given regression points (training data), and is defined for values in between (unseed data), will interpolate it. Maybe you think about linear interpolation specifically, but there are many types of interpolation, and any mathematical function, any neural network is just another form of interpolation.

Interpolation is also related to extrapolation. In higher dimensional spaces, the distinction is not so clear. In terms of machine learning, you would call this generalization.

The question is more, is it a good type of interpolation/extrapolation/generalization. You measure that on a test set.

And mathematically speaking, your brain also is just doing another type of interpolation/extrapolation.

Post reply on HN